-- Logs begin at Tue 2022-08-30 00:17:18 UTC, end at Tue 2022-08-30 01:21:08 UTC. -- Aug 30 00:47:01.222184 dv-cinder-815146-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 00:47:01 2022] *** Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: nodename: dv-cinder-815146-13-iscsi Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: machine: x86_64 Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: clock source: unix Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: pcre jit disabled Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: detected number of CPU cores: 4 Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: current working directory: / Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: detected binary path: /usr/bin/uwsgi-core Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: your processes number limit is 47680 Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: your memory page size is 4096 bytes Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: detected max file descriptor number: 2048 Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: lock engine: pthread robust mutexes Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: thunder lock: enabled Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 00:47:01.236719 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 00:47:01.276652 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: Python main interpreter initialized at 0x563f300ca9c0 Aug 30 00:47:01.276652 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: python threads support enabled Aug 30 00:47:01.276652 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: your server socket listen backlog is limited to 100 connections Aug 30 00:47:01.276652 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: your mercy for graceful operations on workers is 90 seconds Aug 30 00:47:01.277067 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 00:47:01.277173 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: *** Operational MODE: preforking *** Aug 30 00:47:01.277441 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: *** uWSGI is running in multiple interpreter mode *** Aug 30 00:47:01.277441 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: spawned uWSGI master process (pid: 81430) Aug 30 00:47:01.277658 dv-cinder-815146-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 00:47:01.278249 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: spawned uWSGI worker 1 (pid: 81442, cores: 1) Aug 30 00:47:01.278631 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: spawned uWSGI worker 2 (pid: 81443, cores: 1) Aug 30 00:47:01.278788 dv-cinder-815146-13-iscsi devstack@keystone.service[81430]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 00:47:05.573851 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:47:05.573851 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: from cryptography.utils import int_from_bytes Aug 30 00:47:05.574617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:47:05.574617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: from cryptography.utils import int_from_bytes Aug 30 00:47:05.575041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:47:05.575041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: from cryptography.utils import int_from_bytes Aug 30 00:47:05.575149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:47:05.575149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: from cryptography.utils import int_from_bytes Aug 30 00:47:05.623617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 00:47:05.623809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 00:47:05.623979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] command line args: [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 00:47:05.624243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 00:47:05.624332 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 00:47:05.624404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ================================================================================ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 00:47:05.624485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 00:47:05.624610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] command line args: [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 00:47:05.624695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] admin_token = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.624877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 00:47:05.624963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] config_dir = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.625045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ================================================================================ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 00:47:05.625678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.625889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] admin_token = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.625978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] config_source = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.626046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] debug = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.626110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] config_dir = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.626208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.626521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.626630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.626734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] config_source = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.626830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.626933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] debug = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_config_append = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.627970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_dir = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_config_append = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_file = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_dir = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_file = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.628986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.629179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.629262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.629429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.629631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.629724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.629852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.629932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.630064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.630142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.630265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.630423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.630501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.630711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.630864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.630985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.631105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.631226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.631347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.631475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.631598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.631730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.631851 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.631976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.632086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.632200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.632464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.632582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.632723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] publish_errors = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.632840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.632962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.633197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.633377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.633570 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.633689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.633816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] publish_errors = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.633926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.634063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] standard_threads = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.634179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.634314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.634437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.634553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.634667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.634781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.634907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_journal = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.635028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.635144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_json = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.635257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.635376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_stderr = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.635376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_syslog = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.635658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] standard_threads = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.635778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.635893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.636012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.636012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.636227 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.636351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.636467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.636579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.636693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_journal = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.636806 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.636932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_json = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.637059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.637210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_stderr = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.637336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.637451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] use_syslog = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.637565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.637678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:05.637804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.637921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.638034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.638148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.638267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.638382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.638507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.638622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.638736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.638856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.638971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.639085 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.639210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.639331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.639447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.639565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.639719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.639837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.639963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.640080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.640200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.640316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.640429 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.640548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.640673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.641457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.641591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.641591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.641754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.641815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.641955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.642075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.642192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.642306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.642420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.642544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.642661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.642781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.642897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.643010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.643124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.643247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.643363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.643483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.643598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.643711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.643830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.643952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.644068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.644181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.644297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.644410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.644524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.644649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.644770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.644885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.645005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.645143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.645260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.645387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.645503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.645617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.645730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.645843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.645956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.646078 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.646193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.646308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.646422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.646535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.646648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.646769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.646885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.646998 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.647112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.647225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.647345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.647541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.647664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.647780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.647902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.648023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.648139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.648262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.648383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.648499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.648613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.648726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.648829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.648935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.649032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.649150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.649248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.649343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.649447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.649556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.649654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.649791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.649911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.650034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.650151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.650274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.650435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.650550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.650698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.650830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.650958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.651075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.651189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.651312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.651424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.651538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.651659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.651786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.651909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.652025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.652146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.652261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.652387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.652504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.652504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.652719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.652833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.652948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.653058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.653206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.653327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.653444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.653565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.653682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.653797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.653912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.654034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.654151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.654265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.654379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.654502 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.654619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.654742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.654859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.654978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.655093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.655207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.655321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.655444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.655560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.655674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.655796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.655918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.656033 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.656156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.656272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.656386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.656501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.656501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.656699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.656699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.656964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.657063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.657220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.657220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.657436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.657566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.657683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.657798 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.657914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.658029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.658143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.658265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.658382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.658496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.658610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.658725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.658839 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.659036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.659161 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.659278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.659398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.659514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.659641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.659769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.659935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.660053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.660053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.660269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.660384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.660498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.660498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.660729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.660845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.660960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.661089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.661207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.661336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.661454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.661569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.661684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.661799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.661919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.external = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.662044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.662162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.662277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.662399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.662554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.662672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.662817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.662947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.password = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.663072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.663191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.token = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.663306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.663434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.663551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.663667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.663791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.663914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.664032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.external = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.664149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.664264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.664381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.664505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.664624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.664740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.664859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.664975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.password = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.665133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.665266 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] auth.token = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.665385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.665501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.665628 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.665747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.665863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.665986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.666109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.666226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.666342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.666458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.666574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.666698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.666815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.666930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.667045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.667160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.667275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.667399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.667516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.667632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.667748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.667863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.667978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.668102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.668219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.668336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.668452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.668567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.668683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.668807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.668931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.669048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.669177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.669294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.669411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.669539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.669658 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.669773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.669925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.670045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.670161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.670286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.670403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.670520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.670636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.670751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.670875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.670993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.671109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.671224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.671340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.671456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.671580 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.671699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.671814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.671930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.672055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.672171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.672280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.672379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.672475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.672573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.672689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.672805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.672996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.673130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.673248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.673364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.673480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.673609 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.673609 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.673840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.673840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.674064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.674182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.674297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.674421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.674538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.674655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.674771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.674892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.675009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.675133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.675252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.675368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.675484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.675599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.675716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.675841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.675959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.676075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.676191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.676306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.676428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.676554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.676671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.676788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.676904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.677020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.677185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.677316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.677435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.677550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.677665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.677782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.677898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.678022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.678140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.678255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.678403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.678524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.678640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.678764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.678883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.679005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.679122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.679265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.679383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.679508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.679626 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.679741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.679856 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.680008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.680127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.680251 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.680369 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.680484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.680599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.680715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.680838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.680957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.681096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.681216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.681343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.681461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.681584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.681702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.681818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.681934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.682050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.682165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.682289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.682407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.682522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.682637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.682753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.682869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.682992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.683111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.683226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.683342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.683457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.683572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.683696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.683815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.683930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.684045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.684160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.684276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.684399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.684517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.684633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.684752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.684867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.684983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.685154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.685275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.685391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.685506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.685622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.685738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.685862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.685979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.686095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.686211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.686327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.686442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.686566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.686683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.686799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.686914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.687029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.687144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.687269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.687386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.687503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.688111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.688229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.688328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.688434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.688534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.688638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.688755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.688871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.688987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.689132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.689252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.689369 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.689484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.689599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.689715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.689840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.689957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.690110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.690230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.690347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.690463 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.690587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.690587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.690806 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.690922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.691037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.691153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.691276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.691394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.691511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.691626 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.691742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.691858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.691982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.692100 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.692215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.692331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.692447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.692563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.692691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.692810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.692925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.693343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.693466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.693584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.693713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.693833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.693949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.694065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.694065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.694282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.694282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.694510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.694627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.694746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.694862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.694978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.695103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.695220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.695336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.695452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.695568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.695685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.695809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.695927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.696044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] role.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.696166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.696283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] role.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.696400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.696591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.696712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.696829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.696945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.697061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.697210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.697329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.697446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.697562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.697678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.697794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.697918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] role.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.698036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.698151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] role.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.698267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.698383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.698498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.698622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.698741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.698857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.698973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.699088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.699204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.699329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.699447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.699563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.699679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.699796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.699796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.700022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.700176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.700295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.700411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.700527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.700643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.701122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.701250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.701368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.701484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.701601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.701717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.701846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.701846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.702068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.702184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.702299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.702414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.702538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.702656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.702773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.702888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.703005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.703129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.703317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.703982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.704104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.704222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.704322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.704483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.704620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.704742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.704859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.704975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.705111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.705246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.705365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.705482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.705598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.705714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.705838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.705964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.706083 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.706200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.706315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.706432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.706548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.706548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.706841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.706965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.707089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.707208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.707323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.707448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.707566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.707682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.707798 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.707913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.708029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.708153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.708272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.708389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.708505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.708622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.708738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.708862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.708980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.709115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.709237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.709366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.709493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.709611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.709735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.709853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.709968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.710084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.710239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.710360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.710483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.710600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.710715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.710831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.710954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.711079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.711203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.711320 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.711443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.711561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.711675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.711790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.711904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.712019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.712143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.712260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.712375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.712491 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.712605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.712720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.712843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.712961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.713126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.713245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.713360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.713476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.713607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.713725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.713841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.713955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 00:47:05.714663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:05.714663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81443) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 00:47:05.740634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.740860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.740947 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.741100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.741189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.741340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.749517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.749610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.778475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.778688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.778780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.778937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.833189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.834365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.881814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.882042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.882133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:05.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.039650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.039878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.040102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.040322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.040537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.040747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.040830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.040993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.041241 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.041460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.041676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.041888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.063438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.063641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.063727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.063893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.063974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.064087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.064169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.064298 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.064389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.064509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.064591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.064714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.064795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.064924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.075961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.076342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.089340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.089453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.089587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.089679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.089754 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.089826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.089934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.090016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.090151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.090234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.090364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.090445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.090573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81443) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.090664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:06.107692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.107906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:47:06.107990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.108152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:47:06.108233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.108432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:47:06.108515 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.108675 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:47:06.108757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.108949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.109040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.109236 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.109612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.109709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.109784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.109846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.119106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.119343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.119427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.119622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.119703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.119888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.119971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.120167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.126904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.127157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.129809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.130068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.130150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.130359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.130440 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.130645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.130737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.130929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.131009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.131189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.131275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.131461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.131542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.131732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.131813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.132007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.132099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.132280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.132360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.132550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.132630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.132815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.155714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.155919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.157942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.158049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.158226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.158336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.162750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.162889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.163035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.163172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.163315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.163451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.169821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.169907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.170107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.170190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.174207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.174326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.174498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.174616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.174781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.174899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.175064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.175181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.175338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.175456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.175730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.175997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.176265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.177286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.177575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.177856 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.194051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.194154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.194325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.194431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.198218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.198317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.201436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.201522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.201742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.201824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.206689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.206789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.209830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.209916 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.212951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.213047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.213273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.213359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.213558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.213642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.213849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.213933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.214132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.214215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.214433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.214524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.230581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.230866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.230950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.231129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.231212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.231374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.231456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.231704 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.238379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.239083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.240025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.240548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.241606 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.241883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.242128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.242407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.244515 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.244802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.245028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.245130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.245341 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.245437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.245653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.245741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.246010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.246298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.257684 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.257970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.258202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.258286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.258485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.258567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.258785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.258875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.259062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.259143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.259353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.259433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.259655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.259935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.275305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.275895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.278336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.278809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.281224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.281938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.282801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.283173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.284285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.284571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.284776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.284858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.285062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.285375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.291338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.291769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.292812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.293268 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.295701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.295999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.296246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.296340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.296537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.296619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.296822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.297170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.304661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.304955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.305037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.305262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.305346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.305542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.312144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81443) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.312426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:06.312526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.313816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.313925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.314005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.314075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.314355 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.314447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.314630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.314723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.314913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.315012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.315189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.315290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.315467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.315549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.315744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.315832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.316019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.316100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.316283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.316363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.316543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.316630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.316808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.316897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81443) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.317102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:06.351486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563f300ca9c0 pid: 81443 (default app) Aug 30 00:47:06.352075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563f300ca9c0 pid: 81442 (default app) Aug 30 00:47:06.354883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725aa1f3-ea54-457a-b2f4-a44f56e9fdd4 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:06.355168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725aa1f3-ea54-457a-b2f4-a44f56e9fdd4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:06.355437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725aa1f3-ea54-457a-b2f4-a44f56e9fdd4 None None] PATH_INFO: `/v3` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:06.359659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1/1] 10.222.0.78 () {50 vars in 791 bytes} [Tue Aug 30 00:47:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:12.992592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c29502-f142-4ea2-b369-894b75ba39ed None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:12.993146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c29502-f142-4ea2-b369-894b75ba39ed None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:12.993146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c29502-f142-4ea2-b369-894b75ba39ed None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:12.998404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1/2] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:13.004637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:13.005114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:13.005254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:13.075258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: INFO dbcounter [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] Registered counter for database keystone Aug 30 00:47:13.095068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG oslo_db.sqlalchemy.engines [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81443) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 00:47:13.101203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writer thread running {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 00:47:13.281139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG passlib.handlers.bcrypt [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81443) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 00:47:13.281592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG passlib.handlers.bcrypt [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81443) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 00:47:13.341018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:13.342645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:13.472785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 2/3] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 00:47:13.932605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c29502-f142-4ea2-b369-894b75ba39ed None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:14.002653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: INFO dbcounter [None req-17c29502-f142-4ea2-b369-894b75ba39ed None None] Registered counter for database keystone Aug 30 00:47:14.024996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG oslo_db.sqlalchemy.engines [None req-17c29502-f142-4ea2-b369-894b75ba39ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81442) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 00:47:14.031160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writer thread running {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 00:47:14.150021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c29502-f142-4ea2-b369-894b75ba39ed None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:14.154331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67544b20-483d-4352-a186-ece4de7ee2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:14.154959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67544b20-483d-4352-a186-ece4de7ee2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:14.155239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67544b20-483d-4352-a186-ece4de7ee2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:14.155509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67544b20-483d-4352-a186-ece4de7ee2c1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:14.163887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67544b20-483d-4352-a186-ece4de7ee2c1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:14.215658 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WARNING keystone.common.rbac_enforcer.enforcer [None req-67544b20-483d-4352-a186-ece4de7ee2c1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 00:47:14.215938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67544b20-483d-4352-a186-ece4de7ee2c1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:14.225792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 2/4] 10.222.0.78 () {58 vars in 1324 bytes} [Tue Aug 30 00:47:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:47:14.231845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:14.246535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90a191-8785-4d0a-aa9c-b086fadfb214 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:14.251598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a9832-cc6e-497b-b9d6-2d3d6907eae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:14.252686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a9832-cc6e-497b-b9d6-2d3d6907eae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:14.252963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a9832-cc6e-497b-b9d6-2d3d6907eae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:14.253270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a9832-cc6e-497b-b9d6-2d3d6907eae4 admin admin] PATH_INFO: `/v3/projects/761e332e13aa43bb851c08b247de0d02` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:14.254847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a9832-cc6e-497b-b9d6-2d3d6907eae4 admin admin] RBAC: Authorizing `identity:get_project(project_id=761e332e13aa43bb851c08b247de0d02)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:14.307079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c81a9832-cc6e-497b-b9d6-2d3d6907eae4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 00:47:14.307431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a9832-cc6e-497b-b9d6-2d3d6907eae4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:14.312015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 3/5] 10.222.0.78 () {56 vars in 1207 bytes} [Tue Aug 30 00:47:14 2022] GET /identity/v3/projects/761e332e13aa43bb851c08b247de0d02 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:16.198942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4198ac-2aed-4e33-b911-2ca670a63c45 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:16.199515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4198ac-2aed-4e33-b911-2ca670a63c45 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:16.199515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4198ac-2aed-4e33-b911-2ca670a63c45 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:16.203582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 3/6] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:47:16.209666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb94223-525c-4690-b680-ee773bdb6725 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:16.210108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb94223-525c-4690-b680-ee773bdb6725 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:16.210202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb94223-525c-4690-b680-ee773bdb6725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:16.229745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8cb94223-525c-4690-b680-ee773bdb6725 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:16.231255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8cb94223-525c-4690-b680-ee773bdb6725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:16.239324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 4/7] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 00:47:16.504380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4198ac-2aed-4e33-b911-2ca670a63c45 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:16.516530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4198ac-2aed-4e33-b911-2ca670a63c45 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:16.520783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b0366-3c52-425e-ac57-2bda8328f4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:16.521394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b0366-3c52-425e-ac57-2bda8328f4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:16.521671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b0366-3c52-425e-ac57-2bda8328f4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:16.521939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b0366-3c52-425e-ac57-2bda8328f4a4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:16.529708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b0366-3c52-425e-ac57-2bda8328f4a4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:16.531437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b0366-3c52-425e-ac57-2bda8328f4a4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:16.540273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 4/8] 10.222.0.78 () {58 vars in 1324 bytes} [Tue Aug 30 00:47:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:47:16.545647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb94223-525c-4690-b680-ee773bdb6725 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:16.555196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb94223-525c-4690-b680-ee773bdb6725 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:16.559559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea7008-79f9-4e1f-8ed0-b122cba844a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:16.560182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea7008-79f9-4e1f-8ed0-b122cba844a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:16.560464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea7008-79f9-4e1f-8ed0-b122cba844a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:16.560738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea7008-79f9-4e1f-8ed0-b122cba844a0 admin admin] PATH_INFO: `/v3/users/d9331e55306b44e2a9e5ecea982fbc2f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:16.562080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea7008-79f9-4e1f-8ed0-b122cba844a0 admin admin] RBAC: Authorizing `identity:get_user(user_id=d9331e55306b44e2a9e5ecea982fbc2f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:16.564571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea7008-79f9-4e1f-8ed0-b122cba844a0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:16.566139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 5/9] 10.222.0.78 () {56 vars in 1198 bytes} [Tue Aug 30 00:47:16 2022] GET /identity/v3/users/d9331e55306b44e2a9e5ecea982fbc2f => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:18.533231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6808f8a-3206-42f0-ac98-29a36a42770c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:18.533768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6808f8a-3206-42f0-ac98-29a36a42770c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:18.533768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6808f8a-3206-42f0-ac98-29a36a42770c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:18.535217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 5/10] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:18.540062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd331b-e04a-485a-9f6c-1806dd021685 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:18.540498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd331b-e04a-485a-9f6c-1806dd021685 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:18.540593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd331b-e04a-485a-9f6c-1806dd021685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:18.553694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344510c-f625-44c2-8085-9e76b00461a6 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:18.553988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344510c-f625-44c2-8085-9e76b00461a6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:18.554259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344510c-f625-44c2-8085-9e76b00461a6 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:18.555542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 6/11] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:47:18.559508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-79bd331b-e04a-485a-9f6c-1806dd021685 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:18.560442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0adda0-5bb7-4346-861e-0ae88653b44b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:18.560706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-79bd331b-e04a-485a-9f6c-1806dd021685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:18.560828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0adda0-5bb7-4346-861e-0ae88653b44b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:18.560986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0adda0-5bb7-4346-861e-0ae88653b44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:18.602303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG passlib.handlers.bcrypt [None req-3b0adda0-5bb7-4346-861e-0ae88653b44b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81442) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 00:47:18.602694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG passlib.handlers.bcrypt [None req-3b0adda0-5bb7-4346-861e-0ae88653b44b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81442) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 00:47:18.641452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 6/12] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:18.651988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3b0adda0-5bb7-4346-861e-0ae88653b44b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:18.655222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3b0adda0-5bb7-4346-861e-0ae88653b44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:18.756954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 7/13] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:47:18.909131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd331b-e04a-485a-9f6c-1806dd021685 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:18.933891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:19.014872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd331b-e04a-485a-9f6c-1806dd021685 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:19.028543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0adda0-5bb7-4346-861e-0ae88653b44b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:19.050637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaca9b-4548-436f-b12a-63d4bd558952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:19.051274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaca9b-4548-436f-b12a-63d4bd558952 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:19.051554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaca9b-4548-436f-b12a-63d4bd558952 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:19.051826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaca9b-4548-436f-b12a-63d4bd558952 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:19.062744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abaca9b-4548-436f-b12a-63d4bd558952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-9abaca9b-4548-436f-b12a-63d4bd558952 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:19.072367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:19.073510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:19.074486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 7/14] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:19.081232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaca9b-4548-436f-b12a-63d4bd558952 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:19.130635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0adda0-5bb7-4346-861e-0ae88653b44b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:19.132871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:19.166504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:19.167322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39927cb1-7bf7-4e1f-8971-1ac77742ca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:19.168196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39927cb1-7bf7-4e1f-8971-1ac77742ca86 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:19.168457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39927cb1-7bf7-4e1f-8971-1ac77742ca86 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:19.168721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39927cb1-7bf7-4e1f-8971-1ac77742ca86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:19.186517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaca9b-4548-436f-b12a-63d4bd558952 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:19.220303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30434-1902-4f93-b742-687f0272833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:19.220917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30434-1902-4f93-b742-687f0272833d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:19.221244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30434-1902-4f93-b742-687f0272833d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:19.221518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30434-1902-4f93-b742-687f0272833d None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:19.222182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30434-1902-4f93-b742-687f0272833d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:19.222482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30434-1902-4f93-b742-687f0272833d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:19.223884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30434-1902-4f93-b742-687f0272833d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:19.236554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 8/15] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:19.237154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39927cb1-7bf7-4e1f-8971-1ac77742ca86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:19.238781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39927cb1-7bf7-4e1f-8971-1ac77742ca86 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:19.242717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30434-1902-4f93-b742-687f0272833d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:19.339940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 8/16] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:47:19.343292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30434-1902-4f93-b742-687f0272833d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:19.346217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39927cb1-7bf7-4e1f-8971-1ac77742ca86 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:19.349742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:19.378795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c37180-05f2-430d-a987-410cd9096ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:19.379417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c37180-05f2-430d-a987-410cd9096ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:19.379694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c37180-05f2-430d-a987-410cd9096ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:19.379963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c37180-05f2-430d-a987-410cd9096ec6 None admin] PATH_INFO: `/v3/users/d9331e55306b44e2a9e5ecea982fbc2f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:19.381103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c37180-05f2-430d-a987-410cd9096ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=d9331e55306b44e2a9e5ecea982fbc2f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:19.382552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c37180-05f2-430d-a987-410cd9096ec6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:19.384198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 9/17] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:19 2022] GET /identity/v3/users/d9331e55306b44e2a9e5ecea982fbc2f => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:19.389675 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c37180-05f2-430d-a987-410cd9096ec6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:19.404772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:19.451422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39927cb1-7bf7-4e1f-8971-1ac77742ca86 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:19.485140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc8728-2612-4022-be6c-137bc2b61a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:19.485979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc8728-2612-4022-be6c-137bc2b61a98 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:19.486489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc8728-2612-4022-be6c-137bc2b61a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:19.486489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc8728-2612-4022-be6c-137bc2b61a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:19.491859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c37180-05f2-430d-a987-410cd9096ec6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:19.504715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc8728-2612-4022-be6c-137bc2b61a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:19.506211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc8728-2612-4022-be6c-137bc2b61a98 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-01dc8728-2612-4022-be6c-137bc2b61a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:47:19.524595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:19.525687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:47:19.526650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:47:19.526650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:47:19.526650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:19.526650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 9/18] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:19.528532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d6f80-8e6f-4153-870d-e4543f95429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:19.529192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d6f80-8e6f-4153-870d-e4543f95429a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:19.529473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d6f80-8e6f-4153-870d-e4543f95429a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:19.529741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d6f80-8e6f-4153-870d-e4543f95429a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:19.530849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d6f80-8e6f-4153-870d-e4543f95429a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:19.533658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d6f80-8e6f-4153-870d-e4543f95429a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:19.533758 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc8728-2612-4022-be6c-137bc2b61a98 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:19.535249 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 10/19] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:19.540728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d6f80-8e6f-4153-870d-e4543f95429a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:19.590719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:19.619932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:19.635880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc8728-2612-4022-be6c-137bc2b61a98 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:19.638236 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d6f80-8e6f-4153-870d-e4543f95429a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:19.669540 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc5023-ece7-4846-8136-76204113269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:19.670161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5023-ece7-4846-8136-76204113269b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:19.670433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5023-ece7-4846-8136-76204113269b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:19.670715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5023-ece7-4846-8136-76204113269b None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:19.671357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc5023-ece7-4846-8136-76204113269b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:19.671660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc5023-ece7-4846-8136-76204113269b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:19.671851 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dba022-a4b3-4d6c-b97f-c1e5d45368e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:19.672684 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dba022-a4b3-4d6c-b97f-c1e5d45368e8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:19.672963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dba022-a4b3-4d6c-b97f-c1e5d45368e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:19.673258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dba022-a4b3-4d6c-b97f-c1e5d45368e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:19.673391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc5023-ece7-4846-8136-76204113269b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:19.673977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dba022-a4b3-4d6c-b97f-c1e5d45368e8 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.domain.id=default, user.id=d9331e55306b44e2a9e5ecea982fbc2f) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:19.674282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dba022-a4b3-4d6c-b97f-c1e5d45368e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:19.675697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dba022-a4b3-4d6c-b97f-c1e5d45368e8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:19.682243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 11/20] 10.222.0.78 () {56 vars in 1322 bytes} [Tue Aug 30 00:47:19 2022] GET /identity/v3/role_assignments?user.id=d9331e55306b44e2a9e5ecea982fbc2f&scope.domain.id=default&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:19.691773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 10/21] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:21.853340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fe670-ab11-4d29-b725-a02506ccffd4 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:21.853845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fe670-ab11-4d29-b725-a02506ccffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:21.853941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fe670-ab11-4d29-b725-a02506ccffd4 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:21.855209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 12/22] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:21.860388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75086c25-39a6-438b-a041-ae81b62a89da None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:21.860876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75086c25-39a6-438b-a041-ae81b62a89da None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:21.861140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75086c25-39a6-438b-a041-ae81b62a89da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:21.886649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-75086c25-39a6-438b-a041-ae81b62a89da None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:21.887859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-75086c25-39a6-438b-a041-ae81b62a89da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:21.983339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 11/23] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:22.000117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac87b-0bb3-412c-bec3-5deaf1a36190 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:22.001383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac87b-0bb3-412c-bec3-5deaf1a36190 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:22.001942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac87b-0bb3-412c-bec3-5deaf1a36190 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:22.003463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 13/24] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:22.009000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a967520-60aa-4a72-8660-a40a5cfc8155 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:22.009329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a967520-60aa-4a72-8660-a40a5cfc8155 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:22.009611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a967520-60aa-4a72-8660-a40a5cfc8155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:22.038986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3a967520-60aa-4a72-8660-a40a5cfc8155 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:22.040167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3a967520-60aa-4a72-8660-a40a5cfc8155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:22.099104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:22.127874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 12/25] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:47:22.358784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8e928-0ba6-4188-b368-a58f870f62e7 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:22.359410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8e928-0ba6-4188-b368-a58f870f62e7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:22.360417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8e928-0ba6-4188-b368-a58f870f62e7 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:22.361977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 14/26] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:22.368722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8466af-9e99-421c-ac1f-bf0e43a37f1b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:22.369017 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8466af-9e99-421c-ac1f-bf0e43a37f1b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:22.369324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8466af-9e99-421c-ac1f-bf0e43a37f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:22.394245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-bf8466af-9e99-421c-ac1f-bf0e43a37f1b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:22.395406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bf8466af-9e99-421c-ac1f-bf0e43a37f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:22.487709 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 13/27] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:22.518533 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8e928-0ba6-4188-b368-a58f870f62e7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:22.606570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d47275-32f3-4aae-9110-0b0b654412b0 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:22.607353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d47275-32f3-4aae-9110-0b0b654412b0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:22.607846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d47275-32f3-4aae-9110-0b0b654412b0 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:22.609431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 14/28] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:47:22.615326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092a81-0ea0-4a6a-a248-18c607efe22a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:22.615881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092a81-0ea0-4a6a-a248-18c607efe22a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:22.616371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092a81-0ea0-4a6a-a248-18c607efe22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:22.654001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-bf092a81-0ea0-4a6a-a248-18c607efe22a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:22.655136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bf092a81-0ea0-4a6a-a248-18c607efe22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:22.678845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8e928-0ba6-4188-b368-a58f870f62e7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:22.732375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056a35d-072e-4f44-a476-ecb6a5b83c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:22.733419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056a35d-072e-4f44-a476-ecb6a5b83c37 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:22.733905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056a35d-072e-4f44-a476-ecb6a5b83c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:22.734396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056a35d-072e-4f44-a476-ecb6a5b83c37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:22.748348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056a35d-072e-4f44-a476-ecb6a5b83c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-9056a35d-072e-4f44-a476-ecb6a5b83c37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:22.753746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:22.754569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:22.755322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:22.758104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 15/29] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:47:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:22.759108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056a35d-072e-4f44-a476-ecb6a5b83c37 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:22.764734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:22.843079 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 15/30] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:47:22.844883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092a81-0ea0-4a6a-a248-18c607efe22a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:22.882789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:22.884144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056a35d-072e-4f44-a476-ecb6a5b83c37 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:22.927161 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3112d7-d07b-4be0-b03f-814ed7a454dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:22.928131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3112d7-d07b-4be0-b03f-814ed7a454dd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:22.928611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3112d7-d07b-4be0-b03f-814ed7a454dd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:22.929102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3112d7-d07b-4be0-b03f-814ed7a454dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:22.940205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3112d7-d07b-4be0-b03f-814ed7a454dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:22.942017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3112d7-d07b-4be0-b03f-814ed7a454dd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-2d3112d7-d07b-4be0-b03f-814ed7a454dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:47:22.961131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:22.962013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:47:22.962812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:47:22.962812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:47:22.962812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:22.964986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 16/31] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:22.969813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092a81-0ea0-4a6a-a248-18c607efe22a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:22.970905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3112d7-d07b-4be0-b03f-814ed7a454dd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:23.029057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed342c6e-4cdf-4993-a584-2579290a5821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:23.031024 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed342c6e-4cdf-4993-a584-2579290a5821 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.032640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed342c6e-4cdf-4993-a584-2579290a5821 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.033218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed342c6e-4cdf-4993-a584-2579290a5821 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.035631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed342c6e-4cdf-4993-a584-2579290a5821 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:23.037885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed342c6e-4cdf-4993-a584-2579290a5821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:23.039877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed342c6e-4cdf-4993-a584-2579290a5821 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:23.068106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 16/32] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:23.074736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed342c6e-4cdf-4993-a584-2579290a5821 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:23.083891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:23.173052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3112d7-d07b-4be0-b03f-814ed7a454dd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:23.225348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897bae8-38c2-474f-af89-1382ec3a8e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:23.226829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897bae8-38c2-474f-af89-1382ec3a8e00 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.227482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897bae8-38c2-474f-af89-1382ec3a8e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.228065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897bae8-38c2-474f-af89-1382ec3a8e00 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.229245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897bae8-38c2-474f-af89-1382ec3a8e00 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:23.231542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897bae8-38c2-474f-af89-1382ec3a8e00 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:23.236152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:23.239909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed342c6e-4cdf-4993-a584-2579290a5821 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:23.290642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2243fc-92e8-4961-99f6-558872fa7368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:23.291683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2243fc-92e8-4961-99f6-558872fa7368 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.292216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2243fc-92e8-4961-99f6-558872fa7368 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.292755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2243fc-92e8-4961-99f6-558872fa7368 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.293774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2243fc-92e8-4961-99f6-558872fa7368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:23.294277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2243fc-92e8-4961-99f6-558872fa7368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:23.295938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2243fc-92e8-4961-99f6-558872fa7368 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:23.309905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 17/33] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:23.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2243fc-92e8-4961-99f6-558872fa7368 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:23.373454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 17/34] 10.222.0.78 () {60 vars in 1131 bytes} [Tue Aug 30 00:47:22 2022] POST /identity/v3/roles => generated 225 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:23.374216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897bae8-38c2-474f-af89-1382ec3a8e00 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:23.452766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2243fc-92e8-4961-99f6-558872fa7368 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:23.496651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e44b26-55e8-4e64-8371-12082170ae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:23.497815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e44b26-55e8-4e64-8371-12082170ae4a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.498102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e44b26-55e8-4e64-8371-12082170ae4a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.498376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e44b26-55e8-4e64-8371-12082170ae4a None admin] PATH_INFO: `/v3/users/d9331e55306b44e2a9e5ecea982fbc2f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.500721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e44b26-55e8-4e64-8371-12082170ae4a None admin] RBAC: Authorizing `identity:get_user(user_id=d9331e55306b44e2a9e5ecea982fbc2f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:23.502235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e44b26-55e8-4e64-8371-12082170ae4a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:23.504100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 18/35] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:23 2022] GET /identity/v3/users/d9331e55306b44e2a9e5ecea982fbc2f => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:23.509458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897bae8-38c2-474f-af89-1382ec3a8e00 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:23.509619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e44b26-55e8-4e64-8371-12082170ae4a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:23.531372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:23.544451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:23.567446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e490ea5-bbc8-4325-a498-01db0cd7e61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:23.568074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e490ea5-bbc8-4325-a498-01db0cd7e61c None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.568357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e490ea5-bbc8-4325-a498-01db0cd7e61c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.568634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e490ea5-bbc8-4325-a498-01db0cd7e61c None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.569607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e490ea5-bbc8-4325-a498-01db0cd7e61c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:23.571141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e490ea5-bbc8-4325-a498-01db0cd7e61c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:23.590581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 18/36] 10.222.0.78 () {60 vars in 1131 bytes} [Tue Aug 30 00:47:23 2022] POST /identity/v3/roles => generated 219 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:23.591897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763380f1-6fac-47d7-9f3a-7dd60e0fb565 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.592182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763380f1-6fac-47d7-9f3a-7dd60e0fb565 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.592458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763380f1-6fac-47d7-9f3a-7dd60e0fb565 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.598123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 19/37] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:23.598780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763380f1-6fac-47d7-9f3a-7dd60e0fb565 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:23.657003 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e44b26-55e8-4e64-8371-12082170ae4a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:23.701811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f558a13-0c30-4b6b-a724-32c68fe1da6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:23.702853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f558a13-0c30-4b6b-a724-32c68fe1da6f None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.703457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f558a13-0c30-4b6b-a724-32c68fe1da6f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.704830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f558a13-0c30-4b6b-a724-32c68fe1da6f None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.705757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f558a13-0c30-4b6b-a724-32c68fe1da6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:23.707720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f558a13-0c30-4b6b-a724-32c68fe1da6f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:23.734514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763380f1-6fac-47d7-9f3a-7dd60e0fb565 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:23.777201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2632078-7acb-4203-8ebd-29decaa91559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:23.777851 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2632078-7acb-4203-8ebd-29decaa91559 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.778135 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2632078-7acb-4203-8ebd-29decaa91559 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.778434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2632078-7acb-4203-8ebd-29decaa91559 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.779767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2632078-7acb-4203-8ebd-29decaa91559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:23.781267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2632078-7acb-4203-8ebd-29decaa91559 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:23.789111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 20/38] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:23.789111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1cf15-43cd-47d3-859d-404902ca7f86 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.789111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1cf15-43cd-47d3-859d-404902ca7f86 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.789111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1cf15-43cd-47d3-859d-404902ca7f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.807701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-fea1cf15-43cd-47d3-859d-404902ca7f86 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:23.809293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fea1cf15-43cd-47d3-859d-404902ca7f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:23.856004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 19/39] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:47:23 2022] POST /identity/v3/projects => generated 301 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:23.857487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3f3ff-7689-4947-a652-2b47b1a74fae None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:23.857780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3f3ff-7689-4947-a652-2b47b1a74fae None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:23.858059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3f3ff-7689-4947-a652-2b47b1a74fae None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:23.859611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 20/40] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:23.861248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3f3ff-7689-4947-a652-2b47b1a74fae None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:23.960220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:23.965567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:23.992068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3f3ff-7689-4947-a652-2b47b1a74fae None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:24.005368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 21/41] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:24.005368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4cda5-3ed9-4ca5-a1bf-aa140c362c50 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:24.005368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4cda5-3ed9-4ca5-a1bf-aa140c362c50 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:24.005368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4cda5-3ed9-4ca5-a1bf-aa140c362c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:24.022394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e0f4cda5-3ed9-4ca5-a1bf-aa140c362c50 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:24.023604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e0f4cda5-3ed9-4ca5-a1bf-aa140c362c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:24.046281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d75e2-ed7f-4a38-97aa-8a82485aa51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:24.046755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d75e2-ed7f-4a38-97aa-8a82485aa51e None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:24.047506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d75e2-ed7f-4a38-97aa-8a82485aa51e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:24.047678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d75e2-ed7f-4a38-97aa-8a82485aa51e None admin] PATH_INFO: `/v3/domains/default/users/d9331e55306b44e2a9e5ecea982fbc2f/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:24.051261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d75e2-ed7f-4a38-97aa-8a82485aa51e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d9331e55306b44e2a9e5ecea982fbc2f, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:24.054069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d75e2-ed7f-4a38-97aa-8a82485aa51e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:24.076266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 21/42] 10.222.0.78 () {58 vars in 1361 bytes} [Tue Aug 30 00:47:23 2022] PUT /identity/v3/domains/default/users/d9331e55306b44e2a9e5ecea982fbc2f/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:47:24.114113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 22/43] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:24.290857 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d75e2-ed7f-4a38-97aa-8a82485aa51e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:24.394808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d75e2-ed7f-4a38-97aa-8a82485aa51e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:24.394996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4cda5-3ed9-4ca5-a1bf-aa140c362c50 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:24.431520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc79c5-d3f1-4244-8c17-886e316bc919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:24.432140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc79c5-d3f1-4244-8c17-886e316bc919 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:24.432426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc79c5-d3f1-4244-8c17-886e316bc919 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:24.432704 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc79c5-d3f1-4244-8c17-886e316bc919 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:24.433433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc79c5-d3f1-4244-8c17-886e316bc919 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:24.434863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc79c5-d3f1-4244-8c17-886e316bc919 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:24.446052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:24.457309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 22/44] 10.222.0.78 () {60 vars in 1131 bytes} [Tue Aug 30 00:47:24 2022] POST /identity/v3/roles => generated 223 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:24.504516 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4cda5-3ed9-4ca5-a1bf-aa140c362c50 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:24.507394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:24.544147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb42e2e-0cdf-4269-b4ed-cdc066ff673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:24.544802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb42e2e-0cdf-4269-b4ed-cdc066ff673d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:24.545098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb42e2e-0cdf-4269-b4ed-cdc066ff673d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:24.545392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb42e2e-0cdf-4269-b4ed-cdc066ff673d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:24.546558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb42e2e-0cdf-4269-b4ed-cdc066ff673d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:24.548645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb42e2e-0cdf-4269-b4ed-cdc066ff673d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:24.550489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 23/45] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:24.556074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc79c5-d3f1-4244-8c17-886e316bc919 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:24.670965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc79c5-d3f1-4244-8c17-886e316bc919 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:24.706588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c8270-ce57-42c2-8088-eb05a83e376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:24.707206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c8270-ce57-42c2-8088-eb05a83e376b None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:24.707490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c8270-ce57-42c2-8088-eb05a83e376b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:24.707769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c8270-ce57-42c2-8088-eb05a83e376b None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:24.708422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c8270-ce57-42c2-8088-eb05a83e376b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:24.709894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c8270-ce57-42c2-8088-eb05a83e376b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:24.736672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 23/46] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:47:24 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:25.987282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b25593-a8ac-4df2-9cf8-7e644af745ca None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:25.987796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b25593-a8ac-4df2-9cf8-7e644af745ca None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:25.987907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b25593-a8ac-4df2-9cf8-7e644af745ca None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:25.989222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 24/47] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:25.994411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e337748-56dd-4c9a-b4f7-61f2704ec5e5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:25.994814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e337748-56dd-4c9a-b4f7-61f2704ec5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:25.994964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e337748-56dd-4c9a-b4f7-61f2704ec5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:26.012854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9e337748-56dd-4c9a-b4f7-61f2704ec5e5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:26.014372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9e337748-56dd-4c9a-b4f7-61f2704ec5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:26.061837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:26.085978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 24/48] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:26.352466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b25593-a8ac-4df2-9cf8-7e644af745ca None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:26.454149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b25593-a8ac-4df2-9cf8-7e644af745ca None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:26.487355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b40d4-a65d-4bad-8da2-f41ffd72992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:26.487979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b40d4-a65d-4bad-8da2-f41ffd72992d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:26.488265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b40d4-a65d-4bad-8da2-f41ffd72992d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:26.488539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b40d4-a65d-4bad-8da2-f41ffd72992d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:26.498421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b40d4-a65d-4bad-8da2-f41ffd72992d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-912b40d4-a65d-4bad-8da2-f41ffd72992d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:26.503821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:26.504630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:26.505386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 25/49] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:47:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:26.510812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e337748-56dd-4c9a-b4f7-61f2704ec5e5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:26.611060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e337748-56dd-4c9a-b4f7-61f2704ec5e5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:26.644841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e651-d6d9-4608-a130-6116285fdee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:26.645478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e651-d6d9-4608-a130-6116285fdee2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:26.645761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e651-d6d9-4608-a130-6116285fdee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:26.646037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e651-d6d9-4608-a130-6116285fdee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:26.646705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e651-d6d9-4608-a130-6116285fdee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:26.647011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e651-d6d9-4608-a130-6116285fdee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:26.648380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e651-d6d9-4608-a130-6116285fdee2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:26.657925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 25/50] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:26.663135 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b40d4-a65d-4bad-8da2-f41ffd72992d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:26.664860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc12c14-efe4-4025-a5be-aef02212e0d0 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:26.665174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc12c14-efe4-4025-a5be-aef02212e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:26.665449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc12c14-efe4-4025-a5be-aef02212e0d0 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:26.666729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 26/51] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:26.671552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725e725-e0b6-49ed-8839-0bde74d8078c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:26.671844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725e725-e0b6-49ed-8839-0bde74d8078c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:26.672125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725e725-e0b6-49ed-8839-0bde74d8078c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:26.690535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c725e725-e0b6-49ed-8839-0bde74d8078c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:26.691749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c725e725-e0b6-49ed-8839-0bde74d8078c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:26.709195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:26.730108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:26.762225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 27/52] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:47:26.780958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b40d4-a65d-4bad-8da2-f41ffd72992d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:26.815735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913ad42-88cc-4443-a1a7-2d98344d6578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:26.816356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913ad42-88cc-4443-a1a7-2d98344d6578 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:26.816637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913ad42-88cc-4443-a1a7-2d98344d6578 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:26.816907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913ad42-88cc-4443-a1a7-2d98344d6578 None admin] PATH_INFO: `/v3/users/d9331e55306b44e2a9e5ecea982fbc2f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:26.818051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913ad42-88cc-4443-a1a7-2d98344d6578 None admin] RBAC: Authorizing `identity:get_user(user_id=d9331e55306b44e2a9e5ecea982fbc2f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:26.819753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913ad42-88cc-4443-a1a7-2d98344d6578 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:26.821540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 26/53] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:26 2022] GET /identity/v3/users/d9331e55306b44e2a9e5ecea982fbc2f => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:26.826973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725e725-e0b6-49ed-8839-0bde74d8078c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:27.028728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725e725-e0b6-49ed-8839-0bde74d8078c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:27.040421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913ad42-88cc-4443-a1a7-2d98344d6578 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:27.062362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f727cc-738b-42d2-956e-6cadd2347d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:27.063055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f727cc-738b-42d2-956e-6cadd2347d43 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:27.063830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f727cc-738b-42d2-956e-6cadd2347d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:27.064106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f727cc-738b-42d2-956e-6cadd2347d43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:27.065350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f727cc-738b-42d2-956e-6cadd2347d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:27.066817 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f727cc-738b-42d2-956e-6cadd2347d43 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:27.068368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 28/54] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:27.074262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f727cc-738b-42d2-956e-6cadd2347d43 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:27.115840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:27.138246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:27.141426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913ad42-88cc-4443-a1a7-2d98344d6578 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:27.175303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f727cc-738b-42d2-956e-6cadd2347d43 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:27.176606 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50a1a6-ddaa-494c-a5f7-d9bbf12eed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:27.177268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50a1a6-ddaa-494c-a5f7-d9bbf12eed6a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:27.177558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50a1a6-ddaa-494c-a5f7-d9bbf12eed6a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:27.177827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50a1a6-ddaa-494c-a5f7-d9bbf12eed6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:27.178923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50a1a6-ddaa-494c-a5f7-d9bbf12eed6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:27.180354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50a1a6-ddaa-494c-a5f7-d9bbf12eed6a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:27.181931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 27/55] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:27.187341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50a1a6-ddaa-494c-a5f7-d9bbf12eed6a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:27.209721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c5c56-0e95-4674-be24-df532a9e40fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:27.210328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c5c56-0e95-4674-be24-df532a9e40fd None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:27.210610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c5c56-0e95-4674-be24-df532a9e40fd None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:27.210887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c5c56-0e95-4674-be24-df532a9e40fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:27.211618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c5c56-0e95-4674-be24-df532a9e40fd None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.domain.id=default, user.id=d9331e55306b44e2a9e5ecea982fbc2f) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:27.211936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c5c56-0e95-4674-be24-df532a9e40fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:27.213422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c5c56-0e95-4674-be24-df532a9e40fd None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:27.222184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 29/56] 10.222.0.78 () {56 vars in 1322 bytes} [Tue Aug 30 00:47:27 2022] GET /identity/v3/role_assignments?user.id=d9331e55306b44e2a9e5ecea982fbc2f&scope.domain.id=default&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:27.295261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50a1a6-ddaa-494c-a5f7-d9bbf12eed6a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:27.331480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df22828-7c64-4f36-8045-347b6c025091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:27.332114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22828-7c64-4f36-8045-347b6c025091 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:27.332401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22828-7c64-4f36-8045-347b6c025091 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:27.332680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22828-7c64-4f36-8045-347b6c025091 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:27.333389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df22828-7c64-4f36-8045-347b6c025091 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:27.334850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df22828-7c64-4f36-8045-347b6c025091 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:27.366963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 28/57] 10.222.0.78 () {60 vars in 1140 bytes} [Tue Aug 30 00:47:27 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:47:29.255835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852210f7-4f96-43c6-a2f9-e4a8bbde92fd None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:29.256359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852210f7-4f96-43c6-a2f9-e4a8bbde92fd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:29.256447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852210f7-4f96-43c6-a2f9-e4a8bbde92fd None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:29.258094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 30/58] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:29.263040 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ef542-aea3-4997-b3ce-25677a000267 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:29.263471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ef542-aea3-4997-b3ce-25677a000267 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:29.263599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ef542-aea3-4997-b3ce-25677a000267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:29.283178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d80ef542-aea3-4997-b3ce-25677a000267 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:29.284363 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d80ef542-aea3-4997-b3ce-25677a000267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:29.298294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:29.351141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 29/59] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:29.623447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852210f7-4f96-43c6-a2f9-e4a8bbde92fd None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:29.724293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852210f7-4f96-43c6-a2f9-e4a8bbde92fd None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:29.726539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:29.757070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7f2c2-c46c-469f-a5d9-69da0acc5bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:29.757705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7f2c2-c46c-469f-a5d9-69da0acc5bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:29.757983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7f2c2-c46c-469f-a5d9-69da0acc5bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:29.758254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7f2c2-c46c-469f-a5d9-69da0acc5bc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:29.759355 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7f2c2-c46c-469f-a5d9-69da0acc5bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:29.760779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7f2c2-c46c-469f-a5d9-69da0acc5bc1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:29.762366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 31/60] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:29.768529 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ef542-aea3-4997-b3ce-25677a000267 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:29.871006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ef542-aea3-4997-b3ce-25677a000267 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:29.903113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aec45d-4d00-439c-84c7-c2f9ee198386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:29.903650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aec45d-4d00-439c-84c7-c2f9ee198386 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:29.903928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aec45d-4d00-439c-84c7-c2f9ee198386 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:29.904199 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aec45d-4d00-439c-84c7-c2f9ee198386 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:29.904838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aec45d-4d00-439c-84c7-c2f9ee198386 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:29.906305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aec45d-4d00-439c-84c7-c2f9ee198386 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:29.933505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 30/61] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:47:29 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:32.803398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1f075-586e-4743-88ef-fb6d1d824ae8 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:32.804430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1f075-586e-4743-88ef-fb6d1d824ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:32.804931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1f075-586e-4743-88ef-fb6d1d824ae8 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:32.806572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 32/62] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:32.811935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e0253-bbbe-429b-a5b2-1ca21a0cee12 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:32.812346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e0253-bbbe-429b-a5b2-1ca21a0cee12 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:32.812494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e0253-bbbe-429b-a5b2-1ca21a0cee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:32.833456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:32.842901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f22e0253-bbbe-429b-a5b2-1ca21a0cee12 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:32.844670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f22e0253-bbbe-429b-a5b2-1ca21a0cee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:32.929255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 31/63] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:33.123607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab761b8-3cc9-49fc-95b4-70858dd855b8 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.123607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab761b8-3cc9-49fc-95b4-70858dd855b8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.123607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab761b8-3cc9-49fc-95b4-70858dd855b8 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.123607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 33/64] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:33.129003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a602d97-6e34-4f54-a3d2-686cf5c5883c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.129318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a602d97-6e34-4f54-a3d2-686cf5c5883c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.129590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a602d97-6e34-4f54-a3d2-686cf5c5883c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.154338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4a602d97-6e34-4f54-a3d2-686cf5c5883c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:33.155586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4a602d97-6e34-4f54-a3d2-686cf5c5883c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:33.250303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 32/65] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:33.365454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab761b8-3cc9-49fc-95b4-70858dd855b8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:33.446931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd13d2-609c-42c4-a5df-579a090ef4f1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.447529 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd13d2-609c-42c4-a5df-579a090ef4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.448118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd13d2-609c-42c4-a5df-579a090ef4f1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.453661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 33/66] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:33.455950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7d80-c1cd-451d-9f7e-892172e03ff6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.459918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7d80-c1cd-451d-9f7e-892172e03ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.459918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7d80-c1cd-451d-9f7e-892172e03ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.478081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-80da7d80-c1cd-451d-9f7e-892172e03ff6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:33.479673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-80da7d80-c1cd-451d-9f7e-892172e03ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:33.481006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab761b8-3cc9-49fc-95b4-70858dd855b8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:33.518112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaad06c-c496-4341-9498-dd93527e935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:33.518749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaad06c-c496-4341-9498-dd93527e935d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.519193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaad06c-c496-4341-9498-dd93527e935d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.519685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaad06c-c496-4341-9498-dd93527e935d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.532920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaad06c-c496-4341-9498-dd93527e935d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ecaad06c-c496-4341-9498-dd93527e935d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:33.539936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:33.540796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:33.541585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:33.543987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 34/67] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:33.550783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:33.551611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad0cc2-314f-42b2-866b-ad5ee88dfbf7 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.551611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad0cc2-314f-42b2-866b-ad5ee88dfbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.551873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad0cc2-314f-42b2-866b-ad5ee88dfbf7 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.557200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 35/68] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:33.558159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad0cc2-314f-42b2-866b-ad5ee88dfbf7 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:33.567047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 34/69] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:33.568157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e657e9-38bd-4b12-a5f9-578280f76bc2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.568445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e657e9-38bd-4b12-a5f9-578280f76bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.568721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e657e9-38bd-4b12-a5f9-578280f76bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.602491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:33.605591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-60e657e9-38bd-4b12-a5f9-578280f76bc2 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:33.607403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-60e657e9-38bd-4b12-a5f9-578280f76bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:33.682071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 35/70] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:33.683158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad0cc2-314f-42b2-866b-ad5ee88dfbf7 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:33.687259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d42c7-8095-4dd7-b4d5-cb2cb92009b0 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.687755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d42c7-8095-4dd7-b4d5-cb2cb92009b0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.688854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d42c7-8095-4dd7-b4d5-cb2cb92009b0 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.690493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 36/71] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:33.691985 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d42c7-8095-4dd7-b4d5-cb2cb92009b0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:33.720551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51145d2-f903-4f84-a699-715391156f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:33.721131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51145d2-f903-4f84-a699-715391156f1f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.721436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51145d2-f903-4f84-a699-715391156f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.722020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51145d2-f903-4f84-a699-715391156f1f None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.722425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51145d2-f903-4f84-a699-715391156f1f None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:33.722739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51145d2-f903-4f84-a699-715391156f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:33.724129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51145d2-f903-4f84-a699-715391156f1f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:33.738230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 36/72] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:33.739729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81a954-adbb-403a-a50d-01ea3b592a65 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.739959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81a954-adbb-403a-a50d-01ea3b592a65 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.740243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81a954-adbb-403a-a50d-01ea3b592a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.759311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2b81a954-adbb-403a-a50d-01ea3b592a65 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:33.760627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2b81a954-adbb-403a-a50d-01ea3b592a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:33.826020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d42c7-8095-4dd7-b4d5-cb2cb92009b0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:33.832850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 37/73] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:33.834338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81a954-adbb-403a-a50d-01ea3b592a65 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:33.860419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:33.869931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:33.871972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78576371-fb63-4652-b6bc-cb245bff9349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:33.872770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78576371-fb63-4652-b6bc-cb245bff9349 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:33.873443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78576371-fb63-4652-b6bc-cb245bff9349 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:33.873896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78576371-fb63-4652-b6bc-cb245bff9349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:33.886360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78576371-fb63-4652-b6bc-cb245bff9349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-78576371-fb63-4652-b6bc-cb245bff9349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:33.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:33.892546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:33.893397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:33.895953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 37/74] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:47:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:33.896927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78576371-fb63-4652-b6bc-cb245bff9349 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:33.974195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81a954-adbb-403a-a50d-01ea3b592a65 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.009253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78576371-fb63-4652-b6bc-cb245bff9349 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.009447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c417a9d-83f9-4974-a9f0-dde413ba0657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.010008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c417a9d-83f9-4974-a9f0-dde413ba0657 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.010291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c417a9d-83f9-4974-a9f0-dde413ba0657 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.010564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c417a9d-83f9-4974-a9f0-dde413ba0657 None admin] PATH_INFO: `/v3/users/42f1327f05514bbeb23ec0f553202fb7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.028939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c417a9d-83f9-4974-a9f0-dde413ba0657 None admin] RBAC: Authorizing `identity:get_user(user_id=42f1327f05514bbeb23ec0f553202fb7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.030486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c417a9d-83f9-4974-a9f0-dde413ba0657 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.033007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 38/75] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:33 2022] GET /identity/v3/users/42f1327f05514bbeb23ec0f553202fb7 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:34.038707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c417a9d-83f9-4974-a9f0-dde413ba0657 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.045408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e80c11-8b32-4c6a-ac2e-c9fe9cb33381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.046049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e80c11-8b32-4c6a-ac2e-c9fe9cb33381 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.046335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e80c11-8b32-4c6a-ac2e-c9fe9cb33381 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.046612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e80c11-8b32-4c6a-ac2e-c9fe9cb33381 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.058187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e80c11-8b32-4c6a-ac2e-c9fe9cb33381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-f0e80c11-8b32-4c6a-ac2e-c9fe9cb33381 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:34.063278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:34.064292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:34.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:34.066867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 38/76] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:47:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:34.067791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e80c11-8b32-4c6a-ac2e-c9fe9cb33381 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.118297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:34.133590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:34.151874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c417a9d-83f9-4974-a9f0-dde413ba0657 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.174785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e80c11-8b32-4c6a-ac2e-c9fe9cb33381 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.186058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ee8d0-1247-497c-ba9a-5173e2fa553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.186668 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ee8d0-1247-497c-ba9a-5173e2fa553a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.186953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ee8d0-1247-497c-ba9a-5173e2fa553a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.187227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ee8d0-1247-497c-ba9a-5173e2fa553a None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.187895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ee8d0-1247-497c-ba9a-5173e2fa553a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:34.188984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ee8d0-1247-497c-ba9a-5173e2fa553a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.189728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ee8d0-1247-497c-ba9a-5173e2fa553a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.198073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 39/77] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:34.198867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ee8d0-1247-497c-ba9a-5173e2fa553a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.215694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140edff4-4fe9-44cf-af6b-a0c2240cda9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.216770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140edff4-4fe9-44cf-af6b-a0c2240cda9e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.216770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140edff4-4fe9-44cf-af6b-a0c2240cda9e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.216770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140edff4-4fe9-44cf-af6b-a0c2240cda9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.229556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140edff4-4fe9-44cf-af6b-a0c2240cda9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-140edff4-4fe9-44cf-af6b-a0c2240cda9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:34.234212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:34.235080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:34.236051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 39/78] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:34.237871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140edff4-4fe9-44cf-af6b-a0c2240cda9e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.305060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ee8d0-1247-497c-ba9a-5173e2fa553a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.338975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e437ef-9b8e-4de8-b7db-38d81661f404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.339237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140edff4-4fe9-44cf-af6b-a0c2240cda9e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.339585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e437ef-9b8e-4de8-b7db-38d81661f404 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.339868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e437ef-9b8e-4de8-b7db-38d81661f404 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.340140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e437ef-9b8e-4de8-b7db-38d81661f404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.347247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:34.356088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:34.375198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de7fda-8865-4a40-bf79-a6a2af538414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.375933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de7fda-8865-4a40-bf79-a6a2af538414 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.376133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de7fda-8865-4a40-bf79-a6a2af538414 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.376407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de7fda-8865-4a40-bf79-a6a2af538414 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.377092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de7fda-8865-4a40-bf79-a6a2af538414 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:34.377412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de7fda-8865-4a40-bf79-a6a2af538414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.378815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de7fda-8865-4a40-bf79-a6a2af538414 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.388031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 40/79] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:34.388828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de7fda-8865-4a40-bf79-a6a2af538414 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.416053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e437ef-9b8e-4de8-b7db-38d81661f404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.417575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e437ef-9b8e-4de8-b7db-38d81661f404 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.490066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de7fda-8865-4a40-bf79-a6a2af538414 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.515058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 40/80] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:34.515903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e437ef-9b8e-4de8-b7db-38d81661f404 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.524868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25201385-3415-4e26-a267-1c3596646917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.525526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25201385-3415-4e26-a267-1c3596646917 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.525821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25201385-3415-4e26-a267-1c3596646917 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.526084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25201385-3415-4e26-a267-1c3596646917 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.527210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25201385-3415-4e26-a267-1c3596646917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.529096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25201385-3415-4e26-a267-1c3596646917 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.530665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 41/81] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:34.531427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25201385-3415-4e26-a267-1c3596646917 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.565812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:34.566496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:34.621496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e437ef-9b8e-4de8-b7db-38d81661f404 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.631740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25201385-3415-4e26-a267-1c3596646917 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.655315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b40e5e-727d-40cf-af06-47fb737ed559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.655945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b40e5e-727d-40cf-af06-47fb737ed559 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.656219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b40e5e-727d-40cf-af06-47fb737ed559 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.656489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b40e5e-727d-40cf-af06-47fb737ed559 None admin] PATH_INFO: `/v3/users/d9331e55306b44e2a9e5ecea982fbc2f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.658219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b40e5e-727d-40cf-af06-47fb737ed559 None admin] RBAC: Authorizing `identity:get_user(user_id=d9331e55306b44e2a9e5ecea982fbc2f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.659647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b40e5e-727d-40cf-af06-47fb737ed559 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.661461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 41/82] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/users/d9331e55306b44e2a9e5ecea982fbc2f => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:34.661998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b40e5e-727d-40cf-af06-47fb737ed559 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.670682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef69d2-ac6b-4324-8560-fe6eb09d7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.671289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef69d2-ac6b-4324-8560-fe6eb09d7978 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.671575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef69d2-ac6b-4324-8560-fe6eb09d7978 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.671849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef69d2-ac6b-4324-8560-fe6eb09d7978 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.672556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef69d2-ac6b-4324-8560-fe6eb09d7978 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:34.673456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef69d2-ac6b-4324-8560-fe6eb09d7978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.674836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef69d2-ac6b-4324-8560-fe6eb09d7978 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.682856 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 42/83] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:34.683637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef69d2-ac6b-4324-8560-fe6eb09d7978 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.759479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b40e5e-727d-40cf-af06-47fb737ed559 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.773316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:34.781608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:34.785634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef69d2-ac6b-4324-8560-fe6eb09d7978 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.794418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64466283-74a1-4bab-8ce1-f9c36513ea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.795133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64466283-74a1-4bab-8ce1-f9c36513ea7a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.795483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64466283-74a1-4bab-8ce1-f9c36513ea7a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.795826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64466283-74a1-4bab-8ce1-f9c36513ea7a None admin] PATH_INFO: `/v3/users/42f1327f05514bbeb23ec0f553202fb7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.797040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64466283-74a1-4bab-8ce1-f9c36513ea7a None admin] RBAC: Authorizing `identity:get_user(user_id=42f1327f05514bbeb23ec0f553202fb7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.799093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64466283-74a1-4bab-8ce1-f9c36513ea7a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.800832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 42/84] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/users/42f1327f05514bbeb23ec0f553202fb7 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:34.805928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64466283-74a1-4bab-8ce1-f9c36513ea7a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.819909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032099c6-7a0b-4871-8ada-aa2243667909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.820534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032099c6-7a0b-4871-8ada-aa2243667909 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.820813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032099c6-7a0b-4871-8ada-aa2243667909 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.821099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032099c6-7a0b-4871-8ada-aa2243667909 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.822221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032099c6-7a0b-4871-8ada-aa2243667909 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.823647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032099c6-7a0b-4871-8ada-aa2243667909 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.825290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 43/85] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:34.826114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032099c6-7a0b-4871-8ada-aa2243667909 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.906849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64466283-74a1-4bab-8ce1-f9c36513ea7a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.924558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032099c6-7a0b-4871-8ada-aa2243667909 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:34.940632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ea106-5c28-4d27-a4ee-b87dfdcb4da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.941264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ea106-5c28-4d27-a4ee-b87dfdcb4da7 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.941549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ea106-5c28-4d27-a4ee-b87dfdcb4da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.941815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ea106-5c28-4d27-a4ee-b87dfdcb4da7 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.942463 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ea106-5c28-4d27-a4ee-b87dfdcb4da7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:34.943881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ea106-5c28-4d27-a4ee-b87dfdcb4da7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:34.958343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b637d-e830-42ba-a15c-24f603c1c9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:34.958958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b637d-e830-42ba-a15c-24f603c1c9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.959236 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b637d-e830-42ba-a15c-24f603c1c9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.959511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b637d-e830-42ba-a15c-24f603c1c9b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.973365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 43/86] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:47:34 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:34.974242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ea106-5c28-4d27-a4ee-b87dfdcb4da7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.999552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:35.014407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:35.044117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b637d-e830-42ba-a15c-24f603c1c9b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:35.045667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b637d-e830-42ba-a15c-24f603c1c9b6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:35.096321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ea106-5c28-4d27-a4ee-b87dfdcb4da7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:35.144907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626df86b-f255-44dc-aba4-e700f194d9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:35.145898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626df86b-f255-44dc-aba4-e700f194d9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:35.145898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626df86b-f255-44dc-aba4-e700f194d9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:35.146125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626df86b-f255-44dc-aba4-e700f194d9c9 None admin] PATH_INFO: `/v3/users/42f1327f05514bbeb23ec0f553202fb7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:35.147268 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626df86b-f255-44dc-aba4-e700f194d9c9 None admin] RBAC: Authorizing `identity:get_user(user_id=42f1327f05514bbeb23ec0f553202fb7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:35.148849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626df86b-f255-44dc-aba4-e700f194d9c9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:35.150342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 44/87] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/users/42f1327f05514bbeb23ec0f553202fb7 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:35.151103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626df86b-f255-44dc-aba4-e700f194d9c9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:35.160915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 44/88] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:35.161803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b637d-e830-42ba-a15c-24f603c1c9b6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:35.261922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:35.287512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626df86b-f255-44dc-aba4-e700f194d9c9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:35.290727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:35.306722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b637d-e830-42ba-a15c-24f603c1c9b6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:35.336244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dc336-4b23-4546-b12e-17a258f839c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:35.337183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dc336-4b23-4546-b12e-17a258f839c5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:35.337669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dc336-4b23-4546-b12e-17a258f839c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:35.338100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dc336-4b23-4546-b12e-17a258f839c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:35.347837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a3883-9d20-41c9-87ed-bc9ebcb6ab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:35.348475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a3883-9d20-41c9-87ed-bc9ebcb6ab30 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:35.348763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a3883-9d20-41c9-87ed-bc9ebcb6ab30 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:35.349043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a3883-9d20-41c9-87ed-bc9ebcb6ab30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:35.349857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a3883-9d20-41c9-87ed-bc9ebcb6ab30 None admin] RBAC: Adding query filter params (role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4, user.id=42f1327f05514bbeb23ec0f553202fb7) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:35.350176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a3883-9d20-41c9-87ed-bc9ebcb6ab30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:35.352252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a3883-9d20-41c9-87ed-bc9ebcb6ab30 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:35.364273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 45/89] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:35 2022] GET /identity/v3/role_assignments?user.id=42f1327f05514bbeb23ec0f553202fb7&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:35.364273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a3883-9d20-41c9-87ed-bc9ebcb6ab30 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:35.487645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a3883-9d20-41c9-87ed-bc9ebcb6ab30 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:35.525550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:35.526566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dc336-4b23-4546-b12e-17a258f839c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:35.528133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dc336-4b23-4546-b12e-17a258f839c5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:35.534265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c878841-beac-48ec-ac98-488811af52a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:35.534920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c878841-beac-48ec-ac98-488811af52a0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:35.535208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c878841-beac-48ec-ac98-488811af52a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:35.535487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c878841-beac-48ec-ac98-488811af52a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:35.648719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c878841-beac-48ec-ac98-488811af52a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:35.650248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c878841-beac-48ec-ac98-488811af52a0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:35.703271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 45/90] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:35.708619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dc336-4b23-4546-b12e-17a258f839c5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:35.712301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:35.811785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 46/91] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:35.813090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c878841-beac-48ec-ac98-488811af52a0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:35.854473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:35.866992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dc336-4b23-4546-b12e-17a258f839c5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:35.937258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c878841-beac-48ec-ac98-488811af52a0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:35.951336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031602b-6da0-40a8-9af0-fb97f899130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:35.952509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031602b-6da0-40a8-9af0-fb97f899130d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:35.952940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031602b-6da0-40a8-9af0-fb97f899130d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:35.953380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031602b-6da0-40a8-9af0-fb97f899130d None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:35.955332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031602b-6da0-40a8-9af0-fb97f899130d None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:35.957101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031602b-6da0-40a8-9af0-fb97f899130d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:35.958983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 46/92] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:35 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:35.960145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031602b-6da0-40a8-9af0-fb97f899130d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:36.007691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3508f-ab84-456a-aa7e-311a34ccd21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:36.008740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3508f-ab84-456a-aa7e-311a34ccd21a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:36.009494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3508f-ab84-456a-aa7e-311a34ccd21a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:36.009976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3508f-ab84-456a-aa7e-311a34ccd21a None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:36.011467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3508f-ab84-456a-aa7e-311a34ccd21a None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:36.013365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3508f-ab84-456a-aa7e-311a34ccd21a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:36.015226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 47/93] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:35 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:36.022528 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3508f-ab84-456a-aa7e-311a34ccd21a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:36.101712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:36.157734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031602b-6da0-40a8-9af0-fb97f899130d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:36.173536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:36.180046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3508f-ab84-456a-aa7e-311a34ccd21a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:36.239527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25521975-b4ab-4a03-9777-f8bf938a00c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:36.240086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25521975-b4ab-4a03-9777-f8bf938a00c9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:36.240369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25521975-b4ab-4a03-9777-f8bf938a00c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:36.240643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25521975-b4ab-4a03-9777-f8bf938a00c9 None admin] PATH_INFO: `/v3/projects/995702045b32430d903551357300fd16` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:36.245973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25521975-b4ab-4a03-9777-f8bf938a00c9 None admin] RBAC: Authorizing `identity:get_project(project_id=995702045b32430d903551357300fd16)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:36.247421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25521975-b4ab-4a03-9777-f8bf938a00c9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:36.249682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 47/94] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:35 2022] GET /identity/v3/projects/995702045b32430d903551357300fd16 => generated 312 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:36.250896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25521975-b4ab-4a03-9777-f8bf938a00c9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:36.252736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35543696-8b70-4362-8dd3-c494ab2a86b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:36.253376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35543696-8b70-4362-8dd3-c494ab2a86b1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:36.253656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35543696-8b70-4362-8dd3-c494ab2a86b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:36.253933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35543696-8b70-4362-8dd3-c494ab2a86b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:36.254670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35543696-8b70-4362-8dd3-c494ab2a86b1 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4, user.id=d9331e55306b44e2a9e5ecea982fbc2f) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:36.254979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35543696-8b70-4362-8dd3-c494ab2a86b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:36.258358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35543696-8b70-4362-8dd3-c494ab2a86b1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:36.275365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 48/95] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:36 2022] GET /identity/v3/role_assignments?user.id=d9331e55306b44e2a9e5ecea982fbc2f&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:36.276590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35543696-8b70-4362-8dd3-c494ab2a86b1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:36.496466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35543696-8b70-4362-8dd3-c494ab2a86b1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:36.497449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25521975-b4ab-4a03-9777-f8bf938a00c9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:36.558601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90f376-1364-452f-b270-c4b2a5317a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:36.559719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90f376-1364-452f-b270-c4b2a5317a1c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:36.561373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90f376-1364-452f-b270-c4b2a5317a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:36.562423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90f376-1364-452f-b270-c4b2a5317a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:36.563589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90f376-1364-452f-b270-c4b2a5317a1c None admin] RBAC: Adding query filter params (role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=995702045b32430d903551357300fd16, user.id=42f1327f05514bbeb23ec0f553202fb7) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:36.564268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90f376-1364-452f-b270-c4b2a5317a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:36.566506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90f376-1364-452f-b270-c4b2a5317a1c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:36.573786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618521c3-b4a2-4f11-882d-27092a65388e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:36.574737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618521c3-b4a2-4f11-882d-27092a65388e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:36.575224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618521c3-b4a2-4f11-882d-27092a65388e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:36.576610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618521c3-b4a2-4f11-882d-27092a65388e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:36.577623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618521c3-b4a2-4f11-882d-27092a65388e None admin] RBAC: Adding query filter params (role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4, user.id=42f1327f05514bbeb23ec0f553202fb7) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:36.578330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618521c3-b4a2-4f11-882d-27092a65388e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:36.580808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 49/96] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:36 2022] GET /identity/v3/role_assignments?user.id=42f1327f05514bbeb23ec0f553202fb7&scope.project.id=995702045b32430d903551357300fd16&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:36.582688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618521c3-b4a2-4f11-882d-27092a65388e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:36.589583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 48/97] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:36 2022] GET /identity/v3/role_assignments?user.id=42f1327f05514bbeb23ec0f553202fb7&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 259 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:37.392475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f11c9-8871-447f-8afc-3851b322853f None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:37.392979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f11c9-8871-447f-8afc-3851b322853f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:37.393050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f11c9-8871-447f-8afc-3851b322853f None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:37.394756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 50/98] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:37.400267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979714e-dcfd-4845-9f56-d3503d18c424 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:37.400685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979714e-dcfd-4845-9f56-d3503d18c424 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:37.400898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979714e-dcfd-4845-9f56-d3503d18c424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:37.411937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:37.426102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5979714e-dcfd-4845-9f56-d3503d18c424 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:37.427334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5979714e-dcfd-4845-9f56-d3503d18c424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:37.518726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 49/99] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:37.676021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be8131-e0b0-4bed-acce-f724389cecad None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:37.676330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be8131-e0b0-4bed-acce-f724389cecad None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:37.676588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be8131-e0b0-4bed-acce-f724389cecad None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:37.677966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 51/100] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:37.683365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6eb1d6-f2f5-413b-ba4e-3d93b51dde6e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:37.683895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6eb1d6-f2f5-413b-ba4e-3d93b51dde6e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:37.684363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6eb1d6-f2f5-413b-ba4e-3d93b51dde6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:37.706642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4e6eb1d6-f2f5-413b-ba4e-3d93b51dde6e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:37.708190 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4e6eb1d6-f2f5-413b-ba4e-3d93b51dde6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:37.800683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 50/101] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:37.806178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98117d-a5c9-4bcd-944a-4665da6d123a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:37.806472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98117d-a5c9-4bcd-944a-4665da6d123a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:37.806745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98117d-a5c9-4bcd-944a-4665da6d123a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:37.808197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 52/102] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:37.813144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8da547-88b4-4791-aacd-aa62513bda34 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:37.816824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8da547-88b4-4791-aacd-aa62513bda34 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:37.817136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8da547-88b4-4791-aacd-aa62513bda34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:37.839532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cb8da547-88b4-4791-aacd-aa62513bda34 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:37.840726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cb8da547-88b4-4791-aacd-aa62513bda34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:37.915823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:37.925402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 51/103] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:37.963343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98117d-a5c9-4bcd-944a-4665da6d123a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:38.007394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:38.107041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98117d-a5c9-4bcd-944a-4665da6d123a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:38.146074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb026a-9f47-4257-af09-bb755dd5f4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:38.146711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb026a-9f47-4257-af09-bb755dd5f4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:38.147001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb026a-9f47-4257-af09-bb755dd5f4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:38.147275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb026a-9f47-4257-af09-bb755dd5f4b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:38.161210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb026a-9f47-4257-af09-bb755dd5f4b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-2ebb026a-9f47-4257-af09-bb755dd5f4b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:38.166035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:38.166863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:47:38.167615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:38.168626 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 53/104] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:38.174539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8da547-88b4-4791-aacd-aa62513bda34 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:38.342475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8da547-88b4-4791-aacd-aa62513bda34 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:38.378410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394ffb6-7de9-4606-ac49-f15282343bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:38.379018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394ffb6-7de9-4606-ac49-f15282343bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:38.379297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394ffb6-7de9-4606-ac49-f15282343bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:38.379567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394ffb6-7de9-4606-ac49-f15282343bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:38.380246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394ffb6-7de9-4606-ac49-f15282343bc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:38.380548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394ffb6-7de9-4606-ac49-f15282343bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:38.381963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394ffb6-7de9-4606-ac49-f15282343bc5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:38.390793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 52/105] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:38.402350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb026a-9f47-4257-af09-bb755dd5f4b9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:38.435061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394ffb6-7de9-4606-ac49-f15282343bc5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:38.534638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:38.546304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:38.555762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb026a-9f47-4257-af09-bb755dd5f4b9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:38.602444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3420cd-c1bf-443b-9267-c8dfb10410fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:38.604126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3420cd-c1bf-443b-9267-c8dfb10410fb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:38.604412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3420cd-c1bf-443b-9267-c8dfb10410fb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:38.604688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3420cd-c1bf-443b-9267-c8dfb10410fb None admin] PATH_INFO: `/v3/users/c4baa74808b14b319853e2de7dcac42e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:38.624060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3420cd-c1bf-443b-9267-c8dfb10410fb None admin] RBAC: Authorizing `identity:get_user(user_id=c4baa74808b14b319853e2de7dcac42e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:38.626197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3420cd-c1bf-443b-9267-c8dfb10410fb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:38.628483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 54/106] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:38 2022] GET /identity/v3/users/c4baa74808b14b319853e2de7dcac42e => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:38.634294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3420cd-c1bf-443b-9267-c8dfb10410fb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:38.654008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394ffb6-7de9-4606-ac49-f15282343bc5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:38.726134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5090324-63cc-4d73-85c4-718085bf15ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:38.726786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5090324-63cc-4d73-85c4-718085bf15ba None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:38.727081 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5090324-63cc-4d73-85c4-718085bf15ba None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:38.727362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5090324-63cc-4d73-85c4-718085bf15ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:38.754338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5090324-63cc-4d73-85c4-718085bf15ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-f5090324-63cc-4d73-85c4-718085bf15ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:38.759920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:38.760760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:38.761562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 53/107] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:47:38.768120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3420cd-c1bf-443b-9267-c8dfb10410fb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:38.778076 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5090324-63cc-4d73-85c4-718085bf15ba None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:38.804475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e73b68-e6ba-4f62-80a3-33478fe661d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:38.805489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e73b68-e6ba-4f62-80a3-33478fe661d3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:38.806074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e73b68-e6ba-4f62-80a3-33478fe661d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:38.806547 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e73b68-e6ba-4f62-80a3-33478fe661d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:38.807876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e73b68-e6ba-4f62-80a3-33478fe661d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:38.809580 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e73b68-e6ba-4f62-80a3-33478fe661d3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:38.811390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 55/108] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:38.812421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e73b68-e6ba-4f62-80a3-33478fe661d3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:38.841427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:38.957137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5090324-63cc-4d73-85c4-718085bf15ba None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:38.976271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:38.998735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d0cf-7823-42b0-ab03-8c69a503c70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:39.000033 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d0cf-7823-42b0-ab03-8c69a503c70e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.000639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d0cf-7823-42b0-ab03-8c69a503c70e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.002013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d0cf-7823-42b0-ab03-8c69a503c70e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.017111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e73b68-e6ba-4f62-80a3-33478fe661d3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:39.051768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb3373-889b-4fb4-9bbe-f2b15b4dbdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:39.052476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb3373-889b-4fb4-9bbe-f2b15b4dbdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.052835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb3373-889b-4fb4-9bbe-f2b15b4dbdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.053216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb3373-889b-4fb4-9bbe-f2b15b4dbdc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.053965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb3373-889b-4fb4-9bbe-f2b15b4dbdc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:39.054344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb3373-889b-4fb4-9bbe-f2b15b4dbdc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:39.055804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb3373-889b-4fb4-9bbe-f2b15b4dbdc2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:39.064918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 56/109] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:39.066009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb3373-889b-4fb4-9bbe-f2b15b4dbdc2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:39.145870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:39.165545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d0cf-7823-42b0-ab03-8c69a503c70e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:39.168200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d0cf-7823-42b0-ab03-8c69a503c70e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:39.176592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb3373-889b-4fb4-9bbe-f2b15b4dbdc2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:39.226027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add36c8f-ad36-4c3e-8544-e75b03fabb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:39.226670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add36c8f-ad36-4c3e-8544-e75b03fabb26 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.227024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add36c8f-ad36-4c3e-8544-e75b03fabb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.227372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add36c8f-ad36-4c3e-8544-e75b03fabb26 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.228132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add36c8f-ad36-4c3e-8544-e75b03fabb26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:39.229711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add36c8f-ad36-4c3e-8544-e75b03fabb26 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:39.269540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 57/110] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:47:39 2022] POST /identity/v3/projects => generated 302 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:39.270935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add36c8f-ad36-4c3e-8544-e75b03fabb26 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:39.303630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 54/111] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:39.307943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663369-294a-43b1-82aa-a4820b21dbc2 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.308585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663369-294a-43b1-82aa-a4820b21dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.309104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663369-294a-43b1-82aa-a4820b21dbc2 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.316745 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 55/112] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:39.317949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663369-294a-43b1-82aa-a4820b21dbc2 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:39.356914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:39.406084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add36c8f-ad36-4c3e-8544-e75b03fabb26 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:39.442922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:39.454829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98586902-210f-4396-a19b-16e7e6df13c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:39.456301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98586902-210f-4396-a19b-16e7e6df13c7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.456703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98586902-210f-4396-a19b-16e7e6df13c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.457029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98586902-210f-4396-a19b-16e7e6df13c7 None admin] PATH_INFO: `/v3/users/42f1327f05514bbeb23ec0f553202fb7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.458591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98586902-210f-4396-a19b-16e7e6df13c7 None admin] RBAC: Authorizing `identity:get_user(user_id=42f1327f05514bbeb23ec0f553202fb7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:39.460688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98586902-210f-4396-a19b-16e7e6df13c7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:39.467278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 58/113] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:39 2022] GET /identity/v3/users/42f1327f05514bbeb23ec0f553202fb7 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:39.468907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cf3c2-e80a-4b94-ae64-e1dbd6f1c296 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.469753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cf3c2-e80a-4b94-ae64-e1dbd6f1c296 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.470193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cf3c2-e80a-4b94-ae64-e1dbd6f1c296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.478801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663369-294a-43b1-82aa-a4820b21dbc2 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:39.490677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cd2cf3c2-e80a-4b94-ae64-e1dbd6f1c296 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:39.492123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cd2cf3c2-e80a-4b94-ae64-e1dbd6f1c296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:39.525703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b334277-1ac4-472f-b26e-57146310a02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:39.526345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b334277-1ac4-472f-b26e-57146310a02f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.526633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b334277-1ac4-472f-b26e-57146310a02f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.526909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b334277-1ac4-472f-b26e-57146310a02f None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.528671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b334277-1ac4-472f-b26e-57146310a02f None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:39.530143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b334277-1ac4-472f-b26e-57146310a02f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:39.532380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 56/114] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:39 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:39.535475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7720b-0965-4bd9-a88f-915659be34cc None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.535475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7720b-0965-4bd9-a88f-915659be34cc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.535475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7720b-0965-4bd9-a88f-915659be34cc None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.536427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 57/115] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:39.537177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7720b-0965-4bd9-a88f-915659be34cc None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:39.589694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 59/116] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:39.591111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2f7aa-2e77-4c4e-b2e4-07b43be0e00e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.591461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2f7aa-2e77-4c4e-b2e4-07b43be0e00e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.591799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2f7aa-2e77-4c4e-b2e4-07b43be0e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.616040 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1db2f7aa-2e77-4c4e-b2e4-07b43be0e00e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:39.617824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1db2f7aa-2e77-4c4e-b2e4-07b43be0e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:39.648321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:39.671735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7720b-0965-4bd9-a88f-915659be34cc None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:39.711288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 60/117] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:39.712226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2f7aa-2e77-4c4e-b2e4-07b43be0e00e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:39.733585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa73ef8-373d-49d2-ad35-acb684d83a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:39.734194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73ef8-373d-49d2-ad35-acb684d83a1c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.734469 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73ef8-373d-49d2-ad35-acb684d83a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.734738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73ef8-373d-49d2-ad35-acb684d83a1c None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.738410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa73ef8-373d-49d2-ad35-acb684d83a1c None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:39.739906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa73ef8-373d-49d2-ad35-acb684d83a1c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:39.743426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 58/118] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:39 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:39.744361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9312e-76ea-471d-923c-535de3309633 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.744850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9312e-76ea-471d-923c-535de3309633 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.745149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9312e-76ea-471d-923c-535de3309633 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.747720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 59/119] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:39.752748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9312e-76ea-471d-923c-535de3309633 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:39.819520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:39.902317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2f7aa-2e77-4c4e-b2e4-07b43be0e00e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:39.917154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9312e-76ea-471d-923c-535de3309633 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:39.953124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e94039-36dd-40f7-8c4d-bcb912fd08d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:39.954309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e94039-36dd-40f7-8c4d-bcb912fd08d4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.954683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e94039-36dd-40f7-8c4d-bcb912fd08d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.954966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e94039-36dd-40f7-8c4d-bcb912fd08d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.955699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e94039-36dd-40f7-8c4d-bcb912fd08d4 None admin] RBAC: Adding query filter params (role.id=69d0a4169025430084668f95838669b1, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4, user.id=c4baa74808b14b319853e2de7dcac42e) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:39.956014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e94039-36dd-40f7-8c4d-bcb912fd08d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:39.957533 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e94039-36dd-40f7-8c4d-bcb912fd08d4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:39.961383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7839d2e-3d8d-4904-af88-69a17fb042de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:39.962267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839d2e-3d8d-4904-af88-69a17fb042de None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.962695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839d2e-3d8d-4904-af88-69a17fb042de None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.963114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839d2e-3d8d-4904-af88-69a17fb042de None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/users/42f1327f05514bbeb23ec0f553202fb7/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.963482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 61/120] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:39 2022] GET /identity/v3/role_assignments?user.id=c4baa74808b14b319853e2de7dcac42e&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=69d0a4169025430084668f95838669b1 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:39.965613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7839d2e-3d8d-4904-af88-69a17fb042de None admin] RBAC: Authorizing `identity:create_grant(project_id=09da17aa18cc4293b57f7e462cfd17c4, user_id=42f1327f05514bbeb23ec0f553202fb7, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:39.967411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7839d2e-3d8d-4904-af88-69a17fb042de None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:39.970261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdba42e-f1b5-4b38-a36e-8dd9e41964ca None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:39.970835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdba42e-f1b5-4b38-a36e-8dd9e41964ca None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:39.971191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdba42e-f1b5-4b38-a36e-8dd9e41964ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:39.984022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 60/121] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:47:39 2022] PUT /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/users/42f1327f05514bbeb23ec0f553202fb7/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:47:39.985443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7839d2e-3d8d-4904-af88-69a17fb042de None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.006093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:40.008326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3fdba42e-f1b5-4b38-a36e-8dd9e41964ca None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:40.014356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3fdba42e-f1b5-4b38-a36e-8dd9e41964ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:40.135726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7839d2e-3d8d-4904-af88-69a17fb042de None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:40.193302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 62/122] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:40.194640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdba42e-f1b5-4b38-a36e-8dd9e41964ca None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.204805 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51e29b-96d7-4fc3-b159-539306e2575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:40.205482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51e29b-96d7-4fc3-b159-539306e2575b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:40.205758 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51e29b-96d7-4fc3-b159-539306e2575b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:40.206023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51e29b-96d7-4fc3-b159-539306e2575b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:40.219152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51e29b-96d7-4fc3-b159-539306e2575b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9e51e29b-96d7-4fc3-b159-539306e2575b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:40.225323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:40.226183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:40.226945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 61/123] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:47:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:40.232145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51e29b-96d7-4fc3-b159-539306e2575b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.253257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:40.320403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdba42e-f1b5-4b38-a36e-8dd9e41964ca None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:40.321412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:40.353767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51e29b-96d7-4fc3-b159-539306e2575b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:40.358311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cf89e-df36-4f7b-8a06-49184a301d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:40.359238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cf89e-df36-4f7b-8a06-49184a301d2a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:40.359720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cf89e-df36-4f7b-8a06-49184a301d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:40.360731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cf89e-df36-4f7b-8a06-49184a301d2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:40.372313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53cf89e-df36-4f7b-8a06-49184a301d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c53cf89e-df36-4f7b-8a06-49184a301d2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:40.378221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:40.379143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:40.379904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:40.382547 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 63/124] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:40.388328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cf89e-df36-4f7b-8a06-49184a301d2a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.409859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3fc3-822d-4e14-ac38-fdf0ade68cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:40.410537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3fc3-822d-4e14-ac38-fdf0ade68cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:40.410808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3fc3-822d-4e14-ac38-fdf0ade68cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:40.411083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3fc3-822d-4e14-ac38-fdf0ade68cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:40.411821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d3fc3-822d-4e14-ac38-fdf0ade68cf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:40.412123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d3fc3-822d-4e14-ac38-fdf0ade68cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:40.413652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d3fc3-822d-4e14-ac38-fdf0ade68cf3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:40.423380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 62/125] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:40.428851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3fc3-822d-4e14-ac38-fdf0ade68cf3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.502724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cf89e-df36-4f7b-8a06-49184a301d2a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:40.513402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:40.536920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3fc3-822d-4e14-ac38-fdf0ade68cf3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:40.543801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8fdb1-af0c-4f21-a543-fa0b980aad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:40.544707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8fdb1-af0c-4f21-a543-fa0b980aad03 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:40.545220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8fdb1-af0c-4f21-a543-fa0b980aad03 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:40.545694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8fdb1-af0c-4f21-a543-fa0b980aad03 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:40.546557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8fdb1-af0c-4f21-a543-fa0b980aad03 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:40.547079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8fdb1-af0c-4f21-a543-fa0b980aad03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:40.548920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8fdb1-af0c-4f21-a543-fa0b980aad03 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:40.558564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 64/126] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:40.564519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8fdb1-af0c-4f21-a543-fa0b980aad03 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.580230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9f39b-979b-4acb-bf21-48f29af24780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:40.580920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9f39b-979b-4acb-bf21-48f29af24780 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:40.581230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9f39b-979b-4acb-bf21-48f29af24780 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:40.581502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9f39b-979b-4acb-bf21-48f29af24780 None admin] PATH_INFO: `/v3/users/d9331e55306b44e2a9e5ecea982fbc2f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:40.585787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9f39b-979b-4acb-bf21-48f29af24780 None admin] RBAC: Authorizing `identity:get_user(user_id=d9331e55306b44e2a9e5ecea982fbc2f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:40.587247 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9f39b-979b-4acb-bf21-48f29af24780 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:40.588953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 63/127] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:40 2022] GET /identity/v3/users/d9331e55306b44e2a9e5ecea982fbc2f => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:40.594506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9f39b-979b-4acb-bf21-48f29af24780 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.598668 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:40.676159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8fdb1-af0c-4f21-a543-fa0b980aad03 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:40.724468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a874d-3306-4e23-9243-cdc896a94589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:40.725641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a874d-3306-4e23-9243-cdc896a94589 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:40.726127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a874d-3306-4e23-9243-cdc896a94589 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:40.726651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a874d-3306-4e23-9243-cdc896a94589 None admin] PATH_INFO: `/v3/users/42f1327f05514bbeb23ec0f553202fb7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:40.727999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a874d-3306-4e23-9243-cdc896a94589 None admin] RBAC: Authorizing `identity:get_user(user_id=42f1327f05514bbeb23ec0f553202fb7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:40.729703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a874d-3306-4e23-9243-cdc896a94589 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:40.731508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 65/128] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:40 2022] GET /identity/v3/users/42f1327f05514bbeb23ec0f553202fb7 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:40.732485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a874d-3306-4e23-9243-cdc896a94589 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.767474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9f39b-979b-4acb-bf21-48f29af24780 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:40.781156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:40.817119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeab354-484a-4e23-8043-91733f86e7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:40.817923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeab354-484a-4e23-8043-91733f86e7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:40.818032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeab354-484a-4e23-8043-91733f86e7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:40.818292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeab354-484a-4e23-8043-91733f86e7c7 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:40.819689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeab354-484a-4e23-8043-91733f86e7c7 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:40.821150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeab354-484a-4e23-8043-91733f86e7c7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:40.822963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 64/129] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:40 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:40.823553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeab354-484a-4e23-8043-91733f86e7c7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.852373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a874d-3306-4e23-9243-cdc896a94589 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:40.890828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e0afa-0c8d-4cc9-ba77-8bf94061751c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:40.891854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e0afa-0c8d-4cc9-ba77-8bf94061751c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:40.892419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e0afa-0c8d-4cc9-ba77-8bf94061751c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:40.892892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e0afa-0c8d-4cc9-ba77-8bf94061751c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:40.904415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e0afa-0c8d-4cc9-ba77-8bf94061751c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4c0e0afa-0c8d-4cc9-ba77-8bf94061751c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:40.910672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:40.911519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:40.912281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:40.914866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 66/130] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:47:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:40.915887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e0afa-0c8d-4cc9-ba77-8bf94061751c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:40.950823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:41.010713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeab354-484a-4e23-8043-91733f86e7c7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:41.024440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:41.029208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e0afa-0c8d-4cc9-ba77-8bf94061751c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:41.072159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906920-1781-43b0-b1b9-7bb8bb816067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:41.073188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906920-1781-43b0-b1b9-7bb8bb816067 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.073885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906920-1781-43b0-b1b9-7bb8bb816067 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.075112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906920-1781-43b0-b1b9-7bb8bb816067 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/users/d9331e55306b44e2a9e5ecea982fbc2f/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.078313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906920-1781-43b0-b1b9-7bb8bb816067 None admin] RBAC: Authorizing `identity:create_grant(project_id=09da17aa18cc4293b57f7e462cfd17c4, user_id=d9331e55306b44e2a9e5ecea982fbc2f, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:41.081484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906920-1781-43b0-b1b9-7bb8bb816067 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:41.088526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba2332-c86d-4308-8bc1-f71b4526524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:41.089461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba2332-c86d-4308-8bc1-f71b4526524a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.089938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba2332-c86d-4308-8bc1-f71b4526524a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.090376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba2332-c86d-4308-8bc1-f71b4526524a None admin] PATH_INFO: `/v3/projects/995702045b32430d903551357300fd16` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.092577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba2332-c86d-4308-8bc1-f71b4526524a None admin] RBAC: Authorizing `identity:get_project(project_id=995702045b32430d903551357300fd16)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:41.095362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba2332-c86d-4308-8bc1-f71b4526524a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:41.097662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 65/131] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:40 2022] GET /identity/v3/projects/995702045b32430d903551357300fd16 => generated 312 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:41.098643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba2332-c86d-4308-8bc1-f71b4526524a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:41.110537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 67/132] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:47:40 2022] PUT /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/users/d9331e55306b44e2a9e5ecea982fbc2f/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:47:41.111791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906920-1781-43b0-b1b9-7bb8bb816067 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:41.259910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906920-1781-43b0-b1b9-7bb8bb816067 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:41.300056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8cd9f-c413-4eeb-af60-5048ce8e0f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:41.301210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8cd9f-c413-4eeb-af60-5048ce8e0f9b None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.301724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8cd9f-c413-4eeb-af60-5048ce8e0f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.302256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8cd9f-c413-4eeb-af60-5048ce8e0f9b None admin] PATH_INFO: `/v3/projects/995702045b32430d903551357300fd16/users/42f1327f05514bbeb23ec0f553202fb7/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.308785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8cd9f-c413-4eeb-af60-5048ce8e0f9b None admin] RBAC: Authorizing `identity:create_grant(project_id=995702045b32430d903551357300fd16, user_id=42f1327f05514bbeb23ec0f553202fb7, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:41.310629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8cd9f-c413-4eeb-af60-5048ce8e0f9b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:41.343464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba2332-c86d-4308-8bc1-f71b4526524a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:41.360555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 68/133] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:47:41 2022] PUT /identity/v3/projects/995702045b32430d903551357300fd16/users/42f1327f05514bbeb23ec0f553202fb7/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:47:41.383511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d196f-c205-4819-b7e9-d7c06d581b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:41.384557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d196f-c205-4819-b7e9-d7c06d581b28 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.385063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d196f-c205-4819-b7e9-d7c06d581b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.385567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d196f-c205-4819-b7e9-d7c06d581b28 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.386448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d196f-c205-4819-b7e9-d7c06d581b28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:41.386988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d196f-c205-4819-b7e9-d7c06d581b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:41.388619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d196f-c205-4819-b7e9-d7c06d581b28 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:41.395855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:41.401874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 66/134] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:47:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:41.408079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8cd9f-c413-4eeb-af60-5048ce8e0f9b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:41.445064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:41.546931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8cd9f-c413-4eeb-af60-5048ce8e0f9b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:41.584687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01185b-aea4-4d5d-9b33-0e3a4fbed001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:41.585281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01185b-aea4-4d5d-9b33-0e3a4fbed001 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.585566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01185b-aea4-4d5d-9b33-0e3a4fbed001 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.585847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01185b-aea4-4d5d-9b33-0e3a4fbed001 None admin] PATH_INFO: `/v3/users/42f1327f05514bbeb23ec0f553202fb7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.587053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01185b-aea4-4d5d-9b33-0e3a4fbed001 None admin] RBAC: Authorizing `identity:get_user(user_id=42f1327f05514bbeb23ec0f553202fb7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:41.588640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01185b-aea4-4d5d-9b33-0e3a4fbed001 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:41.590397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 69/135] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:41 2022] GET /identity/v3/users/42f1327f05514bbeb23ec0f553202fb7 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:41.596396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d196f-c205-4819-b7e9-d7c06d581b28 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:41.762785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d196f-c205-4819-b7e9-d7c06d581b28 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:41.807185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01498c92-4ba9-41e3-906c-18c13ec04c48 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.807495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01498c92-4ba9-41e3-906c-18c13ec04c48 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.807750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01498c92-4ba9-41e3-906c-18c13ec04c48 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.810892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 70/136] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:47:41.815286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcbae2-04b9-42e7-85ed-cf4fc5898168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:41.816280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcbae2-04b9-42e7-85ed-cf4fc5898168 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.816509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7607d-d5b5-413d-bf3c-d3ca2fec82cc None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.816799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7607d-d5b5-413d-bf3c-d3ca2fec82cc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.817097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7607d-d5b5-413d-bf3c-d3ca2fec82cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.817395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcbae2-04b9-42e7-85ed-cf4fc5898168 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.817877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcbae2-04b9-42e7-85ed-cf4fc5898168 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.819488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcbae2-04b9-42e7-85ed-cf4fc5898168 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:41.821307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcbae2-04b9-42e7-85ed-cf4fc5898168 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:41.823446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 67/137] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:41 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:41.829753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcbae2-04b9-42e7-85ed-cf4fc5898168 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:41.857205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-29e7607d-d5b5-413d-bf3c-d3ca2fec82cc None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:41.858680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-29e7607d-d5b5-413d-bf3c-d3ca2fec82cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:41.917390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:41.980583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcbae2-04b9-42e7-85ed-cf4fc5898168 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:42.001948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 71/138] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:47:42.037281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80804391-144a-43d4-88a6-4fece5421e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:42.037994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80804391-144a-43d4-88a6-4fece5421e2c None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:42.038287 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80804391-144a-43d4-88a6-4fece5421e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:42.038571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80804391-144a-43d4-88a6-4fece5421e2c None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/users/42f1327f05514bbeb23ec0f553202fb7/roles/26e0da7be50c47de8620dca1c657eb4f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:42.045225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80804391-144a-43d4-88a6-4fece5421e2c None admin] RBAC: Authorizing `identity:create_grant(project_id=09da17aa18cc4293b57f7e462cfd17c4, user_id=42f1327f05514bbeb23ec0f553202fb7, role_id=26e0da7be50c47de8620dca1c657eb4f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:42.046742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80804391-144a-43d4-88a6-4fece5421e2c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:42.082673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 68/139] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:47:41 2022] PUT /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/users/42f1327f05514bbeb23ec0f553202fb7/roles/26e0da7be50c47de8620dca1c657eb4f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:47:42.554449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7607d-d5b5-413d-bf3c-d3ca2fec82cc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:42.578729 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:42.673940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7607d-d5b5-413d-bf3c-d3ca2fec82cc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:42.715659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2278923-7c87-4197-8562-290479027ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:42.716303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2278923-7c87-4197-8562-290479027ebd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:42.716584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2278923-7c87-4197-8562-290479027ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:42.716853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2278923-7c87-4197-8562-290479027ebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:42.718007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2278923-7c87-4197-8562-290479027ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:42.719593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2278923-7c87-4197-8562-290479027ebd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:42.721230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 72/140] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:42.726780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80804391-144a-43d4-88a6-4fece5421e2c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:42.854160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80804391-144a-43d4-88a6-4fece5421e2c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:42.888615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:42.900572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703e960-40f9-429e-94ca-80cfa61d7448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:42.901486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703e960-40f9-429e-94ca-80cfa61d7448 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:42.901958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703e960-40f9-429e-94ca-80cfa61d7448 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:42.902418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703e960-40f9-429e-94ca-80cfa61d7448 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:42.903285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703e960-40f9-429e-94ca-80cfa61d7448 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:42.904919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703e960-40f9-429e-94ca-80cfa61d7448 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:42.946934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 69/141] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:47:42 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:43.200252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b499a9-43d7-4a9c-bb89-1af46a2df3aa None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:43.200618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b499a9-43d7-4a9c-bb89-1af46a2df3aa None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:43.200791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b499a9-43d7-4a9c-bb89-1af46a2df3aa None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:43.202333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 73/142] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:43.205850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf887e-cb25-4195-bf73-1531f97d8e09 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:43.206483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf887e-cb25-4195-bf73-1531f97d8e09 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:43.206951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf887e-cb25-4195-bf73-1531f97d8e09 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:43.208835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 70/143] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:43.211202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d54e3-3d64-4198-a54b-51362ac9ada2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:43.211744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d54e3-3d64-4198-a54b-51362ac9ada2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:43.212254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d54e3-3d64-4198-a54b-51362ac9ada2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:43.225377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d2cf-87db-4964-b37b-cd861502ae9d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:43.226167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d2cf-87db-4964-b37b-cd861502ae9d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:43.226635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d2cf-87db-4964-b37b-cd861502ae9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:43.244736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a11d54e3-3d64-4198-a54b-51362ac9ada2 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:43.250478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a11d54e3-3d64-4198-a54b-51362ac9ada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:43.270075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1e07d2cf-87db-4964-b37b-cd861502ae9d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:43.271270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1e07d2cf-87db-4964-b37b-cd861502ae9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:43.347109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 71/144] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:47:43.365915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 74/145] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:43.684490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07d2cf-87db-4964-b37b-cd861502ae9d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:43.793884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d54e3-3d64-4198-a54b-51362ac9ada2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:43.823465 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:43.826177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07d2cf-87db-4964-b37b-cd861502ae9d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:43.857292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:43.894082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c146835-bb79-4b8e-bc66-2a1fd1abb03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:43.894732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c146835-bb79-4b8e-bc66-2a1fd1abb03d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:43.895023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c146835-bb79-4b8e-bc66-2a1fd1abb03d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:43.895301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c146835-bb79-4b8e-bc66-2a1fd1abb03d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:43.906524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c146835-bb79-4b8e-bc66-2a1fd1abb03d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-5c146835-bb79-4b8e-bc66-2a1fd1abb03d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:43.912794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:43.913674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:43.914457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 72/146] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:43.915830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94a91e-72f5-4072-b9d4-3f05bd1016ba None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:43.915830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94a91e-72f5-4072-b9d4-3f05bd1016ba None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:43.915830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94a91e-72f5-4072-b9d4-3f05bd1016ba None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:43.917955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 73/147] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:43.923357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94a91e-72f5-4072-b9d4-3f05bd1016ba None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:43.981932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d54e3-3d64-4198-a54b-51362ac9ada2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:44.030100 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682511fd-a552-4857-a356-36e3c0021000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:44.030295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682511fd-a552-4857-a356-36e3c0021000 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.030689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682511fd-a552-4857-a356-36e3c0021000 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.031025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682511fd-a552-4857-a356-36e3c0021000 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.044759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682511fd-a552-4857-a356-36e3c0021000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-682511fd-a552-4857-a356-36e3c0021000 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:44.052447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:44.053367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:44.054306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 75/148] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:44.055645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c2ff9-fe9c-49c1-aa84-9b7d1e1691ab None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.055926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c2ff9-fe9c-49c1-aa84-9b7d1e1691ab None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.056195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c2ff9-fe9c-49c1-aa84-9b7d1e1691ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.079003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1b7c2ff9-fe9c-49c1-aa84-9b7d1e1691ab None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:44.079911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1b7c2ff9-fe9c-49c1-aa84-9b7d1e1691ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:44.080125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94a91e-72f5-4072-b9d4-3f05bd1016ba None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:44.119285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cecf0-928a-4933-9fe3-354478366da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:44.120195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cecf0-928a-4933-9fe3-354478366da2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.120643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cecf0-928a-4933-9fe3-354478366da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.121128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cecf0-928a-4933-9fe3-354478366da2 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.121939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cecf0-928a-4933-9fe3-354478366da2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:44.122383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cecf0-928a-4933-9fe3-354478366da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:44.123959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cecf0-928a-4933-9fe3-354478366da2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:44.134290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 74/149] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:44.135299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cecf0-928a-4933-9fe3-354478366da2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:44.154900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 76/150] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:44.155816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c2ff9-fe9c-49c1-aa84-9b7d1e1691ab None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:44.159425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:44.180668 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:44.275130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c2ff9-fe9c-49c1-aa84-9b7d1e1691ab None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:44.283999 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cecf0-928a-4933-9fe3-354478366da2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:44.316447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1248b-27c3-45c0-a96c-6bae7f998bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:44.317611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1248b-27c3-45c0-a96c-6bae7f998bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.318481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1248b-27c3-45c0-a96c-6bae7f998bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.319381 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1248b-27c3-45c0-a96c-6bae7f998bd7 None admin] PATH_INFO: `/v3/users/c4baa74808b14b319853e2de7dcac42e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.320900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1248b-27c3-45c0-a96c-6bae7f998bd7 None admin] RBAC: Authorizing `identity:get_user(user_id=c4baa74808b14b319853e2de7dcac42e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:44.322838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1248b-27c3-45c0-a96c-6bae7f998bd7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:44.323168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e069b-d002-4dad-879d-518653308b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:44.323780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e069b-d002-4dad-879d-518653308b9b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.324061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e069b-d002-4dad-879d-518653308b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.324330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e069b-d002-4dad-879d-518653308b9b None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.325001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e069b-d002-4dad-879d-518653308b9b None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:44.325491 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 77/151] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:44 2022] GET /identity/v3/users/c4baa74808b14b319853e2de7dcac42e => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:44.325611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e069b-d002-4dad-879d-518653308b9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:44.326953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a8a2a-29f9-455f-8bfa-2523a405ad50 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.327063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e069b-d002-4dad-879d-518653308b9b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:44.327513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a8a2a-29f9-455f-8bfa-2523a405ad50 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.327916 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a8a2a-29f9-455f-8bfa-2523a405ad50 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.330224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 78/152] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:44.334923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a8a2a-29f9-455f-8bfa-2523a405ad50 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:44.346205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 75/153] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:44.347921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08698282-0804-4b32-9bf7-32b2e1043a0e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.348382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08698282-0804-4b32-9bf7-32b2e1043a0e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.348854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08698282-0804-4b32-9bf7-32b2e1043a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.372377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-08698282-0804-4b32-9bf7-32b2e1043a0e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:44.374273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-08698282-0804-4b32-9bf7-32b2e1043a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:44.449750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:44.484775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 76/154] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:44.486513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08698282-0804-4b32-9bf7-32b2e1043a0e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:44.492334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a8a2a-29f9-455f-8bfa-2523a405ad50 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:44.522015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:44.546531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc4243-8531-43d8-9d87-819df07a0b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:44.547740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc4243-8531-43d8-9d87-819df07a0b17 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.549299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc4243-8531-43d8-9d87-819df07a0b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.550064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc4243-8531-43d8-9d87-819df07a0b17 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.551591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc4243-8531-43d8-9d87-819df07a0b17 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:44.553809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc4243-8531-43d8-9d87-819df07a0b17 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:44.556427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 79/155] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:44 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:44.562169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc4243-8531-43d8-9d87-819df07a0b17 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:44.654223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08698282-0804-4b32-9bf7-32b2e1043a0e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:44.714666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8d8ec-e536-4f30-a013-0e5086cf04d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:44.715586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8d8ec-e536-4f30-a013-0e5086cf04d7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.715865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8d8ec-e536-4f30-a013-0e5086cf04d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.716141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8d8ec-e536-4f30-a013-0e5086cf04d7 None admin] PATH_INFO: `/v3/users/42f1327f05514bbeb23ec0f553202fb7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.719335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8d8ec-e536-4f30-a013-0e5086cf04d7 None admin] RBAC: Authorizing `identity:get_user(user_id=42f1327f05514bbeb23ec0f553202fb7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:44.721763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8d8ec-e536-4f30-a013-0e5086cf04d7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:44.723586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 77/156] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:44 2022] GET /identity/v3/users/42f1327f05514bbeb23ec0f553202fb7 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:44.729950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8d8ec-e536-4f30-a013-0e5086cf04d7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:44.756911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc4243-8531-43d8-9d87-819df07a0b17 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:44.846478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a75f64-ecff-4c9e-96e6-886c06982240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:44.847096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a75f64-ecff-4c9e-96e6-886c06982240 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.847338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a75f64-ecff-4c9e-96e6-886c06982240 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.847611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a75f64-ecff-4c9e-96e6-886c06982240 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/users/c4baa74808b14b319853e2de7dcac42e/roles/69d0a4169025430084668f95838669b1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.855745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a75f64-ecff-4c9e-96e6-886c06982240 None admin] RBAC: Authorizing `identity:create_grant(project_id=09da17aa18cc4293b57f7e462cfd17c4, user_id=c4baa74808b14b319853e2de7dcac42e, role_id=69d0a4169025430084668f95838669b1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:44.857224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a75f64-ecff-4c9e-96e6-886c06982240 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:44.897634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:44.903290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 80/157] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:47:44 2022] PUT /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/users/c4baa74808b14b319853e2de7dcac42e/roles/69d0a4169025430084668f95838669b1 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:47:44.904207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a75f64-ecff-4c9e-96e6-886c06982240 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:44.905351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:44.916272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8d8ec-e536-4f30-a013-0e5086cf04d7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:44.987926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a77dc7-50f8-43a5-91c1-c140e3eaa22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:44.988550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a77dc7-50f8-43a5-91c1-c140e3eaa22a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.988828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a77dc7-50f8-43a5-91c1-c140e3eaa22a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.989181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a77dc7-50f8-43a5-91c1-c140e3eaa22a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.028208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a75f64-ecff-4c9e-96e6-886c06982240 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:45.066362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49084bc-00f6-42b1-8dd6-c7701042ee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:45.067153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49084bc-00f6-42b1-8dd6-c7701042ee0d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.067541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49084bc-00f6-42b1-8dd6-c7701042ee0d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.067912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49084bc-00f6-42b1-8dd6-c7701042ee0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.080865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49084bc-00f6-42b1-8dd6-c7701042ee0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c49084bc-00f6-42b1-8dd6-c7701042ee0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:45.086830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:45.087750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:45.088495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 81/158] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:47:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:45.089270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49084bc-00f6-42b1-8dd6-c7701042ee0d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:45.111507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a77dc7-50f8-43a5-91c1-c140e3eaa22a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:45.113012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a77dc7-50f8-43a5-91c1-c140e3eaa22a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:45.160257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:45.210909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49084bc-00f6-42b1-8dd6-c7701042ee0d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:45.249928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a6b10-acf9-44d4-a4bb-b574c5d92e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:45.251202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a6b10-acf9-44d4-a4bb-b574c5d92e5a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.251552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a6b10-acf9-44d4-a4bb-b574c5d92e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.251900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a6b10-acf9-44d4-a4bb-b574c5d92e5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.267368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a6b10-acf9-44d4-a4bb-b574c5d92e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:45.267967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-518a6b10-acf9-44d4-a4bb-b574c5d92e5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:45.275391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:45.276357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:45.277196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 82/159] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:45.282286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a6b10-acf9-44d4-a4bb-b574c5d92e5a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:45.284747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 78/160] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:47:45.287458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ee359-8296-4407-9994-9b0eef8d07a5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.288061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ee359-8296-4407-9994-9b0eef8d07a5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.288512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ee359-8296-4407-9994-9b0eef8d07a5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.293892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 79/161] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:45.294754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ee359-8296-4407-9994-9b0eef8d07a5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:45.431283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a6b10-acf9-44d4-a4bb-b574c5d92e5a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:45.453489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ee359-8296-4407-9994-9b0eef8d07a5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:45.454909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:45.471804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaac84-cb16-4a82-a01c-6deda1bc6016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:45.472738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaac84-cb16-4a82-a01c-6deda1bc6016 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.473239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaac84-cb16-4a82-a01c-6deda1bc6016 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.473700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaac84-cb16-4a82-a01c-6deda1bc6016 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.474634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daaac84-cb16-4a82-a01c-6deda1bc6016 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:45.475127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daaac84-cb16-4a82-a01c-6deda1bc6016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:45.476774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daaac84-cb16-4a82-a01c-6deda1bc6016 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:45.486205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 83/162] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:45.487436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaac84-cb16-4a82-a01c-6deda1bc6016 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:45.514684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d44b39-3337-43e7-b3ff-4baf0af058a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:45.515305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d44b39-3337-43e7-b3ff-4baf0af058a8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.515581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d44b39-3337-43e7-b3ff-4baf0af058a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.515855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d44b39-3337-43e7-b3ff-4baf0af058a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.517151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d44b39-3337-43e7-b3ff-4baf0af058a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:45.517460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d44b39-3337-43e7-b3ff-4baf0af058a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:45.518851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d44b39-3337-43e7-b3ff-4baf0af058a8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:45.535254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 80/163] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:45.536715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca19d6-4b14-438e-a87b-1ada8fdf8008 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.536831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca19d6-4b14-438e-a87b-1ada8fdf8008 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.537035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca19d6-4b14-438e-a87b-1ada8fdf8008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.570795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-deca19d6-4b14-438e-a87b-1ada8fdf8008 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:45.572004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-deca19d6-4b14-438e-a87b-1ada8fdf8008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:45.605718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaac84-cb16-4a82-a01c-6deda1bc6016 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:45.649813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba251d-d63e-4570-8315-943c26f45faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:45.650818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba251d-d63e-4570-8315-943c26f45faf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.651296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba251d-d63e-4570-8315-943c26f45faf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.652073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba251d-d63e-4570-8315-943c26f45faf None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.653847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba251d-d63e-4570-8315-943c26f45faf None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:45.656155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba251d-d63e-4570-8315-943c26f45faf None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:45.658735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 84/164] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:45 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:45.665475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba251d-d63e-4570-8315-943c26f45faf None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:45.717672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:45.725042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:45.752671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 81/165] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:45.753559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca19d6-4b14-438e-a87b-1ada8fdf8008 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:45.783861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba251d-d63e-4570-8315-943c26f45faf None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:45.821883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef92749-394c-40ef-bddd-d6010dc10577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:45.822845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef92749-394c-40ef-bddd-d6010dc10577 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.823331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef92749-394c-40ef-bddd-d6010dc10577 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.823789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef92749-394c-40ef-bddd-d6010dc10577 None admin] PATH_INFO: `/v3/users/d9331e55306b44e2a9e5ecea982fbc2f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.825175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef92749-394c-40ef-bddd-d6010dc10577 None admin] RBAC: Authorizing `identity:get_user(user_id=d9331e55306b44e2a9e5ecea982fbc2f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:45.827131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef92749-394c-40ef-bddd-d6010dc10577 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:45.829593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 85/166] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:45 2022] GET /identity/v3/users/d9331e55306b44e2a9e5ecea982fbc2f => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:45.831149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef92749-394c-40ef-bddd-d6010dc10577 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:45.943727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef92749-394c-40ef-bddd-d6010dc10577 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:45.964774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:45.984912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e15994-7b03-415b-9c32-63ec5d348f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:45.987308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e15994-7b03-415b-9c32-63ec5d348f87 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.987787 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e15994-7b03-415b-9c32-63ec5d348f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.988272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e15994-7b03-415b-9c32-63ec5d348f87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.989213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e15994-7b03-415b-9c32-63ec5d348f87 None admin] RBAC: Adding query filter params (role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4, user.id=42f1327f05514bbeb23ec0f553202fb7) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:45.989862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e15994-7b03-415b-9c32-63ec5d348f87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:45.991695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e15994-7b03-415b-9c32-63ec5d348f87 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:45.998767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 86/167] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:45 2022] GET /identity/v3/role_assignments?user.id=42f1327f05514bbeb23ec0f553202fb7&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:45.999906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e15994-7b03-415b-9c32-63ec5d348f87 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:46.023683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca19d6-4b14-438e-a87b-1ada8fdf8008 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:46.120541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ee66c-a6fa-44d6-b959-69c6dff71136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:46.121211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ee66c-a6fa-44d6-b959-69c6dff71136 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:46.121491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ee66c-a6fa-44d6-b959-69c6dff71136 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:46.121815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ee66c-a6fa-44d6-b959-69c6dff71136 None admin] PATH_INFO: `/v3/users/42f1327f05514bbeb23ec0f553202fb7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:46.122550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e15994-7b03-415b-9c32-63ec5d348f87 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:46.122968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ee66c-a6fa-44d6-b959-69c6dff71136 None admin] RBAC: Authorizing `identity:get_user(user_id=42f1327f05514bbeb23ec0f553202fb7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:46.124422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ee66c-a6fa-44d6-b959-69c6dff71136 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:46.126935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 82/168] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:45 2022] GET /identity/v3/users/42f1327f05514bbeb23ec0f553202fb7 => generated 275 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:46.132218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ee66c-a6fa-44d6-b959-69c6dff71136 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:46.164837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f06e-c0c1-4862-8459-c730b392efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:46.166111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f06e-c0c1-4862-8459-c730b392efdc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:46.166647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f06e-c0c1-4862-8459-c730b392efdc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:46.167129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f06e-c0c1-4862-8459-c730b392efdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:46.230021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:46.235002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:46.264827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111f06e-c0c1-4862-8459-c730b392efdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:46.267346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111f06e-c0c1-4862-8459-c730b392efdc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:46.296967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ee66c-a6fa-44d6-b959-69c6dff71136 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:46.346511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e356353-8221-444a-8d7b-7d3fc776ff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:46.347196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e356353-8221-444a-8d7b-7d3fc776ff23 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:46.347509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e356353-8221-444a-8d7b-7d3fc776ff23 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:46.348098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e356353-8221-444a-8d7b-7d3fc776ff23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:46.401229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 87/169] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:46.402431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f06e-c0c1-4862-8459-c730b392efdc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:46.431657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e356353-8221-444a-8d7b-7d3fc776ff23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:46.433217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e356353-8221-444a-8d7b-7d3fc776ff23 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:46.490894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:46.508693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:46.513373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f06e-c0c1-4862-8459-c730b392efdc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:46.554431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4df1f-91a1-4f16-9c01-7507a3c8a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:46.555442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4df1f-91a1-4f16-9c01-7507a3c8a2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:46.555916 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4df1f-91a1-4f16-9c01-7507a3c8a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:46.556389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4df1f-91a1-4f16-9c01-7507a3c8a2b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:46.575464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 83/170] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:46.576324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e356353-8221-444a-8d7b-7d3fc776ff23 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:46.579289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a4df1f-91a1-4f16-9c01-7507a3c8a2b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a0a4df1f-91a1-4f16-9c01-7507a3c8a2b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:46.589107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:46.589986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:46.590853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:46.593195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 88/171] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:47:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:46.594212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4df1f-91a1-4f16-9c01-7507a3c8a2b5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:46.682876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e356353-8221-444a-8d7b-7d3fc776ff23 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:46.720917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f5b6d-be73-467b-b22b-f772fa86c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:46.721580 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f5b6d-be73-467b-b22b-f772fa86c988 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:46.721855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f5b6d-be73-467b-b22b-f772fa86c988 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:46.722119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f5b6d-be73-467b-b22b-f772fa86c988 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:46.723326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f5b6d-be73-467b-b22b-f772fa86c988 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:46.724807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f5b6d-be73-467b-b22b-f772fa86c988 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:46.726378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 84/172] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:46 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:46.727184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f5b6d-be73-467b-b22b-f772fa86c988 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:46.740056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4df1f-91a1-4f16-9c01-7507a3c8a2b5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:46.753642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:46.793052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5658e9-56c6-4ce9-a6ac-16e9018c555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:46.793985 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5658e9-56c6-4ce9-a6ac-16e9018c555e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:46.794421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5658e9-56c6-4ce9-a6ac-16e9018c555e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:46.794730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5658e9-56c6-4ce9-a6ac-16e9018c555e None admin] PATH_INFO: `/v3/projects/995702045b32430d903551357300fd16` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:46.795981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5658e9-56c6-4ce9-a6ac-16e9018c555e None admin] RBAC: Authorizing `identity:get_project(project_id=995702045b32430d903551357300fd16)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:46.797658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5658e9-56c6-4ce9-a6ac-16e9018c555e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:46.799551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 89/173] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:46 2022] GET /identity/v3/projects/995702045b32430d903551357300fd16 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:46.800797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5658e9-56c6-4ce9-a6ac-16e9018c555e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:46.818778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:46.838244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f5b6d-be73-467b-b22b-f772fa86c988 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:46.877858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6774a93-e3ed-476d-bbd2-a6aa20781b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:46.878714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6774a93-e3ed-476d-bbd2-a6aa20781b2d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:46.879154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6774a93-e3ed-476d-bbd2-a6aa20781b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:46.879556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6774a93-e3ed-476d-bbd2-a6aa20781b2d None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:46.880354 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6774a93-e3ed-476d-bbd2-a6aa20781b2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:46.880788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6774a93-e3ed-476d-bbd2-a6aa20781b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:46.882482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6774a93-e3ed-476d-bbd2-a6aa20781b2d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:46.891974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 85/174] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:47:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:46.893594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6774a93-e3ed-476d-bbd2-a6aa20781b2d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:46.943311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5658e9-56c6-4ce9-a6ac-16e9018c555e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:46.989199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:47.002438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f0409-e602-4858-9595-8cd7662370b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:47.003218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f0409-e602-4858-9595-8cd7662370b4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.003723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f0409-e602-4858-9595-8cd7662370b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.004019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f0409-e602-4858-9595-8cd7662370b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.004859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f0409-e602-4858-9595-8cd7662370b4 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4, user.id=d9331e55306b44e2a9e5ecea982fbc2f) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:47.005294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f0409-e602-4858-9595-8cd7662370b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:47.006958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f0409-e602-4858-9595-8cd7662370b4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:47.009928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6774a93-e3ed-476d-bbd2-a6aa20781b2d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:47.013572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 90/175] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:46 2022] GET /identity/v3/role_assignments?user.id=d9331e55306b44e2a9e5ecea982fbc2f&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:47.014326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f0409-e602-4858-9595-8cd7662370b4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:47.050652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a3cca-dfc3-40a0-9035-e1f6f0072a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:47.052041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a3cca-dfc3-40a0-9035-e1f6f0072a86 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.052543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a3cca-dfc3-40a0-9035-e1f6f0072a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.053032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a3cca-dfc3-40a0-9035-e1f6f0072a86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.054173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a3cca-dfc3-40a0-9035-e1f6f0072a86 None admin] RBAC: Adding query filter params (role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=995702045b32430d903551357300fd16, user.id=42f1327f05514bbeb23ec0f553202fb7) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:47.054730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a3cca-dfc3-40a0-9035-e1f6f0072a86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:47.056714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a3cca-dfc3-40a0-9035-e1f6f0072a86 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:47.063994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 86/176] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:46 2022] GET /identity/v3/role_assignments?user.id=42f1327f05514bbeb23ec0f553202fb7&scope.project.id=995702045b32430d903551357300fd16&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:47.130525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:47.158765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f0409-e602-4858-9595-8cd7662370b4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:47.198840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e99d54-af90-4ae4-a3b6-364f8e238d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:47.199504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e99d54-af90-4ae4-a3b6-364f8e238d3f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.199782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e99d54-af90-4ae4-a3b6-364f8e238d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.200049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e99d54-af90-4ae4-a3b6-364f8e238d3f None admin] PATH_INFO: `/v3/users/42f1327f05514bbeb23ec0f553202fb7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.201283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e99d54-af90-4ae4-a3b6-364f8e238d3f None admin] RBAC: Authorizing `identity:get_user(user_id=42f1327f05514bbeb23ec0f553202fb7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:47.202768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e99d54-af90-4ae4-a3b6-364f8e238d3f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:47.205219 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 91/177] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:47 2022] GET /identity/v3/users/42f1327f05514bbeb23ec0f553202fb7 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:47.210833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a3cca-dfc3-40a0-9035-e1f6f0072a86 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:47.366481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a3cca-dfc3-40a0-9035-e1f6f0072a86 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:47.405978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df7b6f-cc20-425c-898e-7beb2285b8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:47.406583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df7b6f-cc20-425c-898e-7beb2285b8ba None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.407458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df7b6f-cc20-425c-898e-7beb2285b8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.407714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df7b6f-cc20-425c-898e-7beb2285b8ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.427469 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:47.459023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f650e-9afe-4d50-936e-962c0928e28e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.459304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f650e-9afe-4d50-936e-962c0928e28e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.459729 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f650e-9afe-4d50-936e-962c0928e28e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.461219 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 92/178] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:47:47.466552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c974a-a0d6-43f2-a643-e5126049dcd7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.466917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c974a-a0d6-43f2-a643-e5126049dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.467678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c974a-a0d6-43f2-a643-e5126049dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.498231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-503c974a-a0d6-43f2-a643-e5126049dcd7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:47.500033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-503c974a-a0d6-43f2-a643-e5126049dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:47.504062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df7b6f-cc20-425c-898e-7beb2285b8ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:47.505784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df7b6f-cc20-425c-898e-7beb2285b8ba None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:47.602856 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 93/179] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:47.629800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 87/180] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:47.635426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c974a-a0d6-43f2-a643-e5126049dcd7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:47.673861 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7e9c7-48c2-4509-b499-794f33f5cc8c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.674206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7e9c7-48c2-4509-b499-794f33f5cc8c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.674449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7e9c7-48c2-4509-b499-794f33f5cc8c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.675859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 88/181] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:47:47.681164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaea85-40fc-45ca-8f0f-f13bc104a79d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.681448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaea85-40fc-45ca-8f0f-f13bc104a79d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.681721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaea85-40fc-45ca-8f0f-f13bc104a79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.717998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d3eaea85-40fc-45ca-8f0f-f13bc104a79d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:47.719206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d3eaea85-40fc-45ca-8f0f-f13bc104a79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:47.765340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:47.807398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c974a-a0d6-43f2-a643-e5126049dcd7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:47.811133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:47.848918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb016a1e-4874-46c4-a18e-2c4b5065ba98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:47.850087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb016a1e-4874-46c4-a18e-2c4b5065ba98 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.851679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb016a1e-4874-46c4-a18e-2c4b5065ba98 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.852711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb016a1e-4874-46c4-a18e-2c4b5065ba98 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.855102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb016a1e-4874-46c4-a18e-2c4b5065ba98 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:47.857999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb016a1e-4874-46c4-a18e-2c4b5065ba98 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:47.861481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 94/182] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:47 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:47.865218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 89/183] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:47:47.867197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb016a1e-4874-46c4-a18e-2c4b5065ba98 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:47.940433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eaea85-40fc-45ca-8f0f-f13bc104a79d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:47.976601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb016a1e-4874-46c4-a18e-2c4b5065ba98 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:48.036536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26222df2-4230-4997-bca7-2a50f3d7ef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:48.037349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26222df2-4230-4997-bca7-2a50f3d7ef2c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.037450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26222df2-4230-4997-bca7-2a50f3d7ef2c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.037689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26222df2-4230-4997-bca7-2a50f3d7ef2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.038419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26222df2-4230-4997-bca7-2a50f3d7ef2c None admin] RBAC: Adding query filter params (role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4, user.id=42f1327f05514bbeb23ec0f553202fb7) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:48.038722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26222df2-4230-4997-bca7-2a50f3d7ef2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:48.040288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26222df2-4230-4997-bca7-2a50f3d7ef2c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:48.052622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 95/184] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:47 2022] GET /identity/v3/role_assignments?user.id=42f1327f05514bbeb23ec0f553202fb7&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:48.085530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eaea85-40fc-45ca-8f0f-f13bc104a79d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:48.131358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa651e-0f88-4c76-b844-a70cf4c3689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:48.131990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa651e-0f88-4c76-b844-a70cf4c3689b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.132265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa651e-0f88-4c76-b844-a70cf4c3689b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.132526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa651e-0f88-4c76-b844-a70cf4c3689b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.149284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa651e-0f88-4c76-b844-a70cf4c3689b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-18fa651e-0f88-4c76-b844-a70cf4c3689b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:48.154104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:48.154957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:48.155726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 90/185] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:47:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:47:48.164193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26222df2-4230-4997-bca7-2a50f3d7ef2c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:48.168289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:48.225848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa651e-0f88-4c76-b844-a70cf4c3689b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:48.313208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26222df2-4230-4997-bca7-2a50f3d7ef2c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:48.313952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:48.352672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c44160-2232-4f13-9362-fb00666fd4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:48.353623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c44160-2232-4f13-9362-fb00666fd4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.354092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c44160-2232-4f13-9362-fb00666fd4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.354507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c44160-2232-4f13-9362-fb00666fd4c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.355333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c44160-2232-4f13-9362-fb00666fd4c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:48.355781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c44160-2232-4f13-9362-fb00666fd4c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:48.357471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c44160-2232-4f13-9362-fb00666fd4c3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:48.359657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa651e-0f88-4c76-b844-a70cf4c3689b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:48.371970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 96/186] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:48.373586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a814f-6528-457e-94e3-db3b3aba66c8 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.374079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a814f-6528-457e-94e3-db3b3aba66c8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.374554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a814f-6528-457e-94e3-db3b3aba66c8 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.376177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 97/187] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:48.377134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a814f-6528-457e-94e3-db3b3aba66c8 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:48.403682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f2be3-5cf9-45f3-9ed1-65daf1088107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:48.403682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f2be3-5cf9-45f3-9ed1-65daf1088107 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.403682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f2be3-5cf9-45f3-9ed1-65daf1088107 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.404282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f2be3-5cf9-45f3-9ed1-65daf1088107 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.418815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f2be3-5cf9-45f3-9ed1-65daf1088107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-f84f2be3-5cf9-45f3-9ed1-65daf1088107 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:48.425310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:48.426229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:48.427612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 91/188] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:47:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:47:48.428274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd9af1-5188-4a67-bf46-a9e18a6250bb None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.428274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd9af1-5188-4a67-bf46-a9e18a6250bb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.428592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd9af1-5188-4a67-bf46-a9e18a6250bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.447207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7bbd9af1-5188-4a67-bf46-a9e18a6250bb None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:48.448387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7bbd9af1-5188-4a67-bf46-a9e18a6250bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:48.466377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:48.509479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a814f-6528-457e-94e3-db3b3aba66c8 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:48.524471 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 92/189] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:48.526816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd9af1-5188-4a67-bf46-a9e18a6250bb None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:48.552405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24392eda-c725-419e-a3fd-5a92fcbd0d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:48.553472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24392eda-c725-419e-a3fd-5a92fcbd0d73 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.553999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24392eda-c725-419e-a3fd-5a92fcbd0d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.554476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24392eda-c725-419e-a3fd-5a92fcbd0d73 None admin] PATH_INFO: `/v3/users/009f8b2c17fa408cb598aefd30ea93ff` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.567102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:48.572014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24392eda-c725-419e-a3fd-5a92fcbd0d73 None admin] RBAC: Authorizing `identity:get_user(user_id=009f8b2c17fa408cb598aefd30ea93ff)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:48.573786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24392eda-c725-419e-a3fd-5a92fcbd0d73 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:48.575713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 98/190] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:48 2022] GET /identity/v3/users/009f8b2c17fa408cb598aefd30ea93ff => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:48.589105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24392eda-c725-419e-a3fd-5a92fcbd0d73 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:48.659258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd9af1-5188-4a67-bf46-a9e18a6250bb None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:48.695301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24392eda-c725-419e-a3fd-5a92fcbd0d73 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:48.701153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae94423-b7d3-4077-8a8b-aee1596ee01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:48.701760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94423-b7d3-4077-8a8b-aee1596ee01b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.702037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94423-b7d3-4077-8a8b-aee1596ee01b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.702303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94423-b7d3-4077-8a8b-aee1596ee01b None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.702966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94423-b7d3-4077-8a8b-aee1596ee01b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:48.703265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94423-b7d3-4077-8a8b-aee1596ee01b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:48.705418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94423-b7d3-4077-8a8b-aee1596ee01b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:48.713401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 93/191] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:47:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:48.714561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b7b98-75ac-4301-8bfe-3473673b05ad None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.714847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b7b98-75ac-4301-8bfe-3473673b05ad None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.715111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b7b98-75ac-4301-8bfe-3473673b05ad None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.716474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 94/192] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:48.718734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b7b98-75ac-4301-8bfe-3473673b05ad None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:48.731140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:48.742581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240fef5-01b6-4491-8015-f2b199fb9c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:48.743184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240fef5-01b6-4491-8015-f2b199fb9c94 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.743463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240fef5-01b6-4491-8015-f2b199fb9c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.743734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240fef5-01b6-4491-8015-f2b199fb9c94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.814262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240fef5-01b6-4491-8015-f2b199fb9c94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:48.814788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:48.815725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240fef5-01b6-4491-8015-f2b199fb9c94 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:48.825712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b7b98-75ac-4301-8bfe-3473673b05ad None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:48.859408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948988b4-1b6f-43ad-9ed6-261d0eac1cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:48.860097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948988b4-1b6f-43ad-9ed6-261d0eac1cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.860375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948988b4-1b6f-43ad-9ed6-261d0eac1cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.860824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948988b4-1b6f-43ad-9ed6-261d0eac1cd8 None admin] PATH_INFO: `/v3/users/009f8b2c17fa408cb598aefd30ea93ff` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.862018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948988b4-1b6f-43ad-9ed6-261d0eac1cd8 None admin] RBAC: Authorizing `identity:get_user(user_id=009f8b2c17fa408cb598aefd30ea93ff)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:48.863441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948988b4-1b6f-43ad-9ed6-261d0eac1cd8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:48.865034 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 95/193] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:48 2022] GET /identity/v3/users/009f8b2c17fa408cb598aefd30ea93ff => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:48.870461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12fb9-f7a1-4f77-afd1-72991e19dde4 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.870750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12fb9-f7a1-4f77-afd1-72991e19dde4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.871026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12fb9-f7a1-4f77-afd1-72991e19dde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.888026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-01f12fb9-f7a1-4f77-afd1-72991e19dde4 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:48.889221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-01f12fb9-f7a1-4f77-afd1-72991e19dde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:48.916773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 99/194] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:48.917569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240fef5-01b6-4491-8015-f2b199fb9c94 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:48.941677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:48.958911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 96/195] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:48.959751 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12fb9-f7a1-4f77-afd1-72991e19dde4 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:49.021356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240fef5-01b6-4491-8015-f2b199fb9c94 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.055339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583711fa-063f-4a6c-a311-8f733008cbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.055987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583711fa-063f-4a6c-a311-8f733008cbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.056235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583711fa-063f-4a6c-a311-8f733008cbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.056504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583711fa-063f-4a6c-a311-8f733008cbd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.057636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583711fa-063f-4a6c-a311-8f733008cbd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:49.059092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583711fa-063f-4a6c-a311-8f733008cbd8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:49.059386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:49.065257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 100/196] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:49.065257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583711fa-063f-4a6c-a311-8f733008cbd8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:49.071211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12fb9-f7a1-4f77-afd1-72991e19dde4 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.108209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60106ec1-9cf5-4227-a9b8-b26f1f0713d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.109274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60106ec1-9cf5-4227-a9b8-b26f1f0713d4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.109561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60106ec1-9cf5-4227-a9b8-b26f1f0713d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.109833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60106ec1-9cf5-4227-a9b8-b26f1f0713d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.162808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:49.173666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583711fa-063f-4a6c-a311-8f733008cbd8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.180101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60106ec1-9cf5-4227-a9b8-b26f1f0713d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:49.181592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60106ec1-9cf5-4227-a9b8-b26f1f0713d4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:49.211574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1c815-ddfe-4324-9300-6ccb0822457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.212204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1c815-ddfe-4324-9300-6ccb0822457e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.212481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1c815-ddfe-4324-9300-6ccb0822457e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.212750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1c815-ddfe-4324-9300-6ccb0822457e None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.214015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1c815-ddfe-4324-9300-6ccb0822457e None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:49.215452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1c815-ddfe-4324-9300-6ccb0822457e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:49.221934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 101/197] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:49 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:49.221934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1c815-ddfe-4324-9300-6ccb0822457e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:49.279572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:49.290020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 97/198] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:49.290827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60106ec1-9cf5-4227-a9b8-b26f1f0713d4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:49.328618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1c815-ddfe-4324-9300-6ccb0822457e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.361945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d3822-0789-408a-bbbd-698b0d0a72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.362543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d3822-0789-408a-bbbd-698b0d0a72d8 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.362818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d3822-0789-408a-bbbd-698b0d0a72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.363085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d3822-0789-408a-bbbd-698b0d0a72d8 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.363728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d3822-0789-408a-bbbd-698b0d0a72d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:49.365200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d3822-0789-408a-bbbd-698b0d0a72d8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:49.386359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 102/199] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:47:49 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:49.387388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d3822-0789-408a-bbbd-698b0d0a72d8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:49.389522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60106ec1-9cf5-4227-a9b8-b26f1f0713d4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.421586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:49.434085 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9662a-eb62-4eec-b7ff-edeef76e82d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.434627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9662a-eb62-4eec-b7ff-edeef76e82d1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.434909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9662a-eb62-4eec-b7ff-edeef76e82d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.435182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9662a-eb62-4eec-b7ff-edeef76e82d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.435940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9662a-eb62-4eec-b7ff-edeef76e82d1 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=b5a0684693f546a9b65d6dd77f36a932, user.id=009f8b2c17fa408cb598aefd30ea93ff) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:49.436252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9662a-eb62-4eec-b7ff-edeef76e82d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:49.437827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9662a-eb62-4eec-b7ff-edeef76e82d1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:49.457839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 98/200] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:49 2022] GET /identity/v3/role_assignments?user.id=009f8b2c17fa408cb598aefd30ea93ff&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:49.459365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9662a-eb62-4eec-b7ff-edeef76e82d1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:49.540488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d3822-0789-408a-bbbd-698b0d0a72d8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.546558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:49.589847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1b92-d53b-4e0a-ab16-a54044601c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.590549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1b92-d53b-4e0a-ab16-a54044601c13 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.590832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1b92-d53b-4e0a-ab16-a54044601c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.591105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1b92-d53b-4e0a-ab16-a54044601c13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.605237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf1b92-d53b-4e0a-ab16-a54044601c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:49.607433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9662a-eb62-4eec-b7ff-edeef76e82d1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-72bf1b92-d53b-4e0a-ab16-a54044601c13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:49.612137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:49.613008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:47:49.613926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:49.614964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 103/201] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:49.626430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1b92-d53b-4e0a-ab16-a54044601c13 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:49.660969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cc145-7dd2-4048-9a6d-04bf6d9c4a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.661642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cc145-7dd2-4048-9a6d-04bf6d9c4a4e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.661969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cc145-7dd2-4048-9a6d-04bf6d9c4a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.662304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cc145-7dd2-4048-9a6d-04bf6d9c4a4e None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.663476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cc145-7dd2-4048-9a6d-04bf6d9c4a4e None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:49.664925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cc145-7dd2-4048-9a6d-04bf6d9c4a4e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:49.666640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 99/202] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:49 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:49.672415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cc145-7dd2-4048-9a6d-04bf6d9c4a4e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:49.726603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:49.750771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1b92-d53b-4e0a-ab16-a54044601c13 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.790491 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a41ec-4233-4550-8660-14a3ad60b748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.790850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cc145-7dd2-4048-9a6d-04bf6d9c4a4e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.791725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a41ec-4233-4550-8660-14a3ad60b748 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.792000 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a41ec-4233-4550-8660-14a3ad60b748 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.792264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a41ec-4233-4550-8660-14a3ad60b748 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.792926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a41ec-4233-4550-8660-14a3ad60b748 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:49.794138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a41ec-4233-4550-8660-14a3ad60b748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:49.795540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a41ec-4233-4550-8660-14a3ad60b748 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:49.807394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 104/203] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:49.813560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a41ec-4233-4550-8660-14a3ad60b748 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:49.822368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:49.838589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0244b7-df10-4ac0-83dc-95d5abf28e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.839312 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0244b7-df10-4ac0-83dc-95d5abf28e99 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.839592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0244b7-df10-4ac0-83dc-95d5abf28e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.839862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0244b7-df10-4ac0-83dc-95d5abf28e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.840597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0244b7-df10-4ac0-83dc-95d5abf28e99 None admin] RBAC: Adding query filter params (role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=b5a0684693f546a9b65d6dd77f36a932, user.id=009f8b2c17fa408cb598aefd30ea93ff) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:49.840908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0244b7-df10-4ac0-83dc-95d5abf28e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:49.842669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0244b7-df10-4ac0-83dc-95d5abf28e99 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:49.851034 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 100/204] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:49 2022] GET /identity/v3/role_assignments?user.id=009f8b2c17fa408cb598aefd30ea93ff&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:49.929911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a41ec-4233-4550-8660-14a3ad60b748 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:49.968904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cc15f-51ab-487b-9865-6aea8b447319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:49.969613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cc15f-51ab-487b-9865-6aea8b447319 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.969888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cc15f-51ab-487b-9865-6aea8b447319 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.970226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cc15f-51ab-487b-9865-6aea8b447319 None admin] PATH_INFO: `/v3/users/c4baa74808b14b319853e2de7dcac42e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.971670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cc15f-51ab-487b-9865-6aea8b447319 None admin] RBAC: Authorizing `identity:get_user(user_id=c4baa74808b14b319853e2de7dcac42e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:49.973177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cc15f-51ab-487b-9865-6aea8b447319 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:49.974809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 105/205] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:49 2022] GET /identity/v3/users/c4baa74808b14b319853e2de7dcac42e => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:49.980766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0244b7-df10-4ac0-83dc-95d5abf28e99 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:50.118305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0244b7-df10-4ac0-83dc-95d5abf28e99 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:50.174865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b0643-405a-45bb-b2d4-8d99e74191a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:50.175485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b0643-405a-45bb-b2d4-8d99e74191a3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:50.175766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b0643-405a-45bb-b2d4-8d99e74191a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:50.176040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b0643-405a-45bb-b2d4-8d99e74191a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:50.265923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:50.287744 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b0643-405a-45bb-b2d4-8d99e74191a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:50.289267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b0643-405a-45bb-b2d4-8d99e74191a3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:50.415465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 101/206] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:50.421744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cc15f-51ab-487b-9865-6aea8b447319 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:50.432402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:50.574819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cc15f-51ab-487b-9865-6aea8b447319 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:50.616503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154f26c-aaac-4d24-8bde-1203130d3f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:50.617170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154f26c-aaac-4d24-8bde-1203130d3f34 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:50.617449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154f26c-aaac-4d24-8bde-1203130d3f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:50.617722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154f26c-aaac-4d24-8bde-1203130d3f34 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:50.618861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154f26c-aaac-4d24-8bde-1203130d3f34 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:50.620305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154f26c-aaac-4d24-8bde-1203130d3f34 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:50.622125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 106/207] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:50 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:50.628828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b0643-405a-45bb-b2d4-8d99e74191a3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:50.777667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:50.794318 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b0643-405a-45bb-b2d4-8d99e74191a3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:50.846572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37ee96-0e02-4b17-895e-8d3317411a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:50.847099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37ee96-0e02-4b17-895e-8d3317411a88 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:50.847378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37ee96-0e02-4b17-895e-8d3317411a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:50.847649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37ee96-0e02-4b17-895e-8d3317411a88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:50.848386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37ee96-0e02-4b17-895e-8d3317411a88 None admin] RBAC: Adding query filter params (role.id=69d0a4169025430084668f95838669b1, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4, user.id=c4baa74808b14b319853e2de7dcac42e) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:50.848696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37ee96-0e02-4b17-895e-8d3317411a88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:50.850154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37ee96-0e02-4b17-895e-8d3317411a88 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:50.862149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 102/208] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:50 2022] GET /identity/v3/role_assignments?user.id=c4baa74808b14b319853e2de7dcac42e&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=69d0a4169025430084668f95838669b1 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:51.655710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c396f-a13e-4fee-8903-4ea35710a442 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:51.656297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c396f-a13e-4fee-8903-4ea35710a442 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:51.656807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c396f-a13e-4fee-8903-4ea35710a442 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:51.659568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 107/209] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:51.670508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d5067-af0b-4a38-870b-b4cb9e789dda None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:51.671005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d5067-af0b-4a38-870b-b4cb9e789dda None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:51.671077 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d5067-af0b-4a38-870b-b4cb9e789dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:51.698578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-404d5067-af0b-4a38-870b-b4cb9e789dda None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:51.699781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-404d5067-af0b-4a38-870b-b4cb9e789dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:51.780443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 103/210] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:47:52.011466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c06b75-5bc6-44c6-803b-2b274594fb1e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.012105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c06b75-5bc6-44c6-803b-2b274594fb1e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.012833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c06b75-5bc6-44c6-803b-2b274594fb1e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.014453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 108/211] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:52.020519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4b2a0-8cad-48bb-ac57-a66c02a425d6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.021069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4b2a0-8cad-48bb-ac57-a66c02a425d6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.021560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4b2a0-8cad-48bb-ac57-a66c02a425d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.043015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ecc4b2a0-8cad-48bb-ac57-a66c02a425d6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:52.044550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ecc4b2a0-8cad-48bb-ac57-a66c02a425d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:52.055054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a547df5-a9b9-49a0-b4f2-22002685750f None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.055411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a547df5-a9b9-49a0-b4f2-22002685750f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.055640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a547df5-a9b9-49a0-b4f2-22002685750f None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.057519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 109/212] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:47:52.062042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a547df5-a9b9-49a0-b4f2-22002685750f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.129994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 104/213] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:52.136560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dbda6-8fd1-4058-964a-67200c157d0e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.136847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dbda6-8fd1-4058-964a-67200c157d0e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.137165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dbda6-8fd1-4058-964a-67200c157d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.165359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a53dbda6-8fd1-4058-964a-67200c157d0e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:52.165583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:52.167110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a53dbda6-8fd1-4058-964a-67200c157d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:52.201460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:52.211291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a547df5-a9b9-49a0-b4f2-22002685750f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:52.272443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b30765-38b8-48ae-8486-4cc0bd51e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:52.273934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b30765-38b8-48ae-8486-4cc0bd51e4ac None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.274423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b30765-38b8-48ae-8486-4cc0bd51e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.274895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b30765-38b8-48ae-8486-4cc0bd51e4ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.283788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 105/214] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:52.289294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9862e28-1bca-4dae-9259-0e2465fd964b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.289294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9862e28-1bca-4dae-9259-0e2465fd964b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.289294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9862e28-1bca-4dae-9259-0e2465fd964b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.289294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 106/215] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:52.296109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a660301-0a74-4f02-af25-4a5214b901a5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.298044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a660301-0a74-4f02-af25-4a5214b901a5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.298912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a660301-0a74-4f02-af25-4a5214b901a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.299455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b30765-38b8-48ae-8486-4cc0bd51e4ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-40b30765-38b8-48ae-8486-4cc0bd51e4ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:52.304815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:52.306128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:52.306921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 110/216] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:47:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:47:52.313022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b30765-38b8-48ae-8486-4cc0bd51e4ac None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.331834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6a660301-0a74-4f02-af25-4a5214b901a5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:52.333781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6a660301-0a74-4f02-af25-4a5214b901a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:52.427261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 107/217] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:52.434882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b30765-38b8-48ae-8486-4cc0bd51e4ac None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:52.442392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a660301-0a74-4f02-af25-4a5214b901a5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.482539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36f069-404a-4431-8b82-decba7b95a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:52.483142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36f069-404a-4431-8b82-decba7b95a96 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.483414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36f069-404a-4431-8b82-decba7b95a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.483679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36f069-404a-4431-8b82-decba7b95a96 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.484695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36f069-404a-4431-8b82-decba7b95a96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:52.485599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36f069-404a-4431-8b82-decba7b95a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:52.487288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36f069-404a-4431-8b82-decba7b95a96 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:52.488955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:52.498477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 111/218] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:52.504368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36f069-404a-4431-8b82-decba7b95a96 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.530542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:52.567131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a660301-0a74-4f02-af25-4a5214b901a5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:52.605468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67669248-ead4-48cb-8623-a4028b0479f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:52.606104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669248-ead4-48cb-8623-a4028b0479f2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.606383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669248-ead4-48cb-8623-a4028b0479f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.606652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669248-ead4-48cb-8623-a4028b0479f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.613020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36f069-404a-4431-8b82-decba7b95a96 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:52.619392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67669248-ead4-48cb-8623-a4028b0479f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-67669248-ead4-48cb-8623-a4028b0479f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:52.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:52.625157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:52.626650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 108/219] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:47:52.627345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67669248-ead4-48cb-8623-a4028b0479f2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.654577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a8340-ed9e-43b0-b468-343e213d06fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:52.655517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a8340-ed9e-43b0-b468-343e213d06fc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.655980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a8340-ed9e-43b0-b468-343e213d06fc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.657408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a8340-ed9e-43b0-b468-343e213d06fc None admin] PATH_INFO: `/v3/users/009f8b2c17fa408cb598aefd30ea93ff` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.658719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a8340-ed9e-43b0-b468-343e213d06fc None admin] RBAC: Authorizing `identity:get_user(user_id=009f8b2c17fa408cb598aefd30ea93ff)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:52.660381 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a8340-ed9e-43b0-b468-343e213d06fc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:52.666609 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 112/220] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:52 2022] GET /identity/v3/users/009f8b2c17fa408cb598aefd30ea93ff => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:52.667303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a8340-ed9e-43b0-b468-343e213d06fc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.730863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:52.735059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67669248-ead4-48cb-8623-a4028b0479f2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:52.769728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:52.774999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a8340-ed9e-43b0-b468-343e213d06fc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:52.775791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bded58-cca3-4821-9a5a-6984b5639f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:52.776427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bded58-cca3-4821-9a5a-6984b5639f3c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.776706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bded58-cca3-4821-9a5a-6984b5639f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.776975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bded58-cca3-4821-9a5a-6984b5639f3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.778128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bded58-cca3-4821-9a5a-6984b5639f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:52.779567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bded58-cca3-4821-9a5a-6984b5639f3c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:52.781265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 109/221] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:52.782501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bded58-cca3-4821-9a5a-6984b5639f3c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.810393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7581abc-e7f8-4cfa-89aa-c9d9e7265a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:52.811025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7581abc-e7f8-4cfa-89aa-c9d9e7265a8e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.811306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7581abc-e7f8-4cfa-89aa-c9d9e7265a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.811592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7581abc-e7f8-4cfa-89aa-c9d9e7265a8e None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.812243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7581abc-e7f8-4cfa-89aa-c9d9e7265a8e None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:52.812552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7581abc-e7f8-4cfa-89aa-c9d9e7265a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:52.814029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7581abc-e7f8-4cfa-89aa-c9d9e7265a8e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:52.822190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 113/222] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:52.822976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7581abc-e7f8-4cfa-89aa-c9d9e7265a8e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.880878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bded58-cca3-4821-9a5a-6984b5639f3c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:52.914267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc3b2b-4af7-4ae4-8f83-5753fadf625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:52.914886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc3b2b-4af7-4ae4-8f83-5753fadf625a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.915161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc3b2b-4af7-4ae4-8f83-5753fadf625a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.915426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc3b2b-4af7-4ae4-8f83-5753fadf625a None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.916523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc3b2b-4af7-4ae4-8f83-5753fadf625a None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:52.917968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc3b2b-4af7-4ae4-8f83-5753fadf625a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:52.919573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 110/223] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:52 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:52.920481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc3b2b-4af7-4ae4-8f83-5753fadf625a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.928958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7581abc-e7f8-4cfa-89aa-c9d9e7265a8e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:52.954272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:52.963470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a59e-f9aa-4a90-9360-c13c571a5cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:52.964084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a59e-f9aa-4a90-9360-c13c571a5cca None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:52.964356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a59e-f9aa-4a90-9360-c13c571a5cca None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:52.964618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a59e-f9aa-4a90-9360-c13c571a5cca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:52.976513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344a59e-f9aa-4a90-9360-c13c571a5cca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b344a59e-f9aa-4a90-9360-c13c571a5cca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:52.981781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:52.982678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 114/224] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:47:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:52.983668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a59e-f9aa-4a90-9360-c13c571a5cca None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:52.990093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:53.023682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc3b2b-4af7-4ae4-8f83-5753fadf625a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.057388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7133565-2f6e-49b4-909d-d19d8300ed3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.057918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7133565-2f6e-49b4-909d-d19d8300ed3b None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.058195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7133565-2f6e-49b4-909d-d19d8300ed3b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.058467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7133565-2f6e-49b4-909d-d19d8300ed3b None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.059116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7133565-2f6e-49b4-909d-d19d8300ed3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:53.060552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7133565-2f6e-49b4-909d-d19d8300ed3b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:53.081255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 111/225] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:47:52 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:53.082039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7133565-2f6e-49b4-909d-d19d8300ed3b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:53.090771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a59e-f9aa-4a90-9360-c13c571a5cca None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.134857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aac1ae-a990-4dbc-9f39-a56e0d09ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.135520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aac1ae-a990-4dbc-9f39-a56e0d09ba5e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.135799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aac1ae-a990-4dbc-9f39-a56e0d09ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.136075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aac1ae-a990-4dbc-9f39-a56e0d09ba5e None admin] PATH_INFO: `/v3/users/90556558521e4a8aab42c948f0552637` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.151724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aac1ae-a990-4dbc-9f39-a56e0d09ba5e None admin] RBAC: Authorizing `identity:get_user(user_id=90556558521e4a8aab42c948f0552637)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:53.153269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aac1ae-a990-4dbc-9f39-a56e0d09ba5e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:53.155198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 115/226] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:52 2022] GET /identity/v3/users/90556558521e4a8aab42c948f0552637 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:53.155788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aac1ae-a990-4dbc-9f39-a56e0d09ba5e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:53.199334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7133565-2f6e-49b4-909d-d19d8300ed3b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.211101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:53.229508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:53.244914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de3f85-684d-4c35-aa77-8f2f0acb9dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.245849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de3f85-684d-4c35-aa77-8f2f0acb9dc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.246288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de3f85-684d-4c35-aa77-8f2f0acb9dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.246791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de3f85-684d-4c35-aa77-8f2f0acb9dc9 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932/users/009f8b2c17fa408cb598aefd30ea93ff/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.249575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de3f85-684d-4c35-aa77-8f2f0acb9dc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5a0684693f546a9b65d6dd77f36a932, user_id=009f8b2c17fa408cb598aefd30ea93ff, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:53.251680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de3f85-684d-4c35-aa77-8f2f0acb9dc9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:53.266761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 112/227] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:47:53 2022] PUT /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932/users/009f8b2c17fa408cb598aefd30ea93ff/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:47:53.267777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de3f85-684d-4c35-aa77-8f2f0acb9dc9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:53.290426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aac1ae-a990-4dbc-9f39-a56e0d09ba5e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.347539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b39fe-e130-4a87-a47d-29e3cc3296e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.347539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b39fe-e130-4a87-a47d-29e3cc3296e1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.347539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b39fe-e130-4a87-a47d-29e3cc3296e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.347539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b39fe-e130-4a87-a47d-29e3cc3296e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.347959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b39fe-e130-4a87-a47d-29e3cc3296e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:53.348919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b39fe-e130-4a87-a47d-29e3cc3296e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:53.350526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b39fe-e130-4a87-a47d-29e3cc3296e1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:53.364441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 116/228] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:47:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:53.371218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b39fe-e130-4a87-a47d-29e3cc3296e1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:53.407697 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de3f85-684d-4c35-aa77-8f2f0acb9dc9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.446626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fb496-e7e1-44e4-961c-89df2a511bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.447239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fb496-e7e1-44e4-961c-89df2a511bbe None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.447517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fb496-e7e1-44e4-961c-89df2a511bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.447789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fb496-e7e1-44e4-961c-89df2a511bbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.492894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b39fe-e130-4a87-a47d-29e3cc3296e1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.500302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:53.513567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:53.523128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fb496-e7e1-44e4-961c-89df2a511bbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:53.524687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fb496-e7e1-44e4-961c-89df2a511bbe None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:53.530140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c81a61-4d63-429f-8726-4d99c396b27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.530777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81a61-4d63-429f-8726-4d99c396b27b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.531049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81a61-4d63-429f-8726-4d99c396b27b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.531313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81a61-4d63-429f-8726-4d99c396b27b None admin] PATH_INFO: `/v3/users/009f8b2c17fa408cb598aefd30ea93ff` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.533184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c81a61-4d63-429f-8726-4d99c396b27b None admin] RBAC: Authorizing `identity:get_user(user_id=009f8b2c17fa408cb598aefd30ea93ff)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:53.534668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c81a61-4d63-429f-8726-4d99c396b27b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:53.536791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 117/229] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:53 2022] GET /identity/v3/users/009f8b2c17fa408cb598aefd30ea93ff => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:53.544255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c81a61-4d63-429f-8726-4d99c396b27b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:53.658044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 113/230] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:53.664525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fb496-e7e1-44e4-961c-89df2a511bbe None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:53.680865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c81a61-4d63-429f-8726-4d99c396b27b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.723644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748cbfb-5bf1-4728-be97-5b6b5b0d2bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.724644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748cbfb-5bf1-4728-be97-5b6b5b0d2bbb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.725301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748cbfb-5bf1-4728-be97-5b6b5b0d2bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.726858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748cbfb-5bf1-4728-be97-5b6b5b0d2bbb None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.728946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748cbfb-5bf1-4728-be97-5b6b5b0d2bbb None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:53.731186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748cbfb-5bf1-4728-be97-5b6b5b0d2bbb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:53.737693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 118/231] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:53 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:53.742338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748cbfb-5bf1-4728-be97-5b6b5b0d2bbb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:53.806696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:53.840855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:53.859998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fb496-e7e1-44e4-961c-89df2a511bbe None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.901464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808f9b4-4672-4eaf-82fe-80c524d0115e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.902332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808f9b4-4672-4eaf-82fe-80c524d0115e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.903447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808f9b4-4672-4eaf-82fe-80c524d0115e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.905165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808f9b4-4672-4eaf-82fe-80c524d0115e None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.908250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808f9b4-4672-4eaf-82fe-80c524d0115e None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:53.911908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808f9b4-4672-4eaf-82fe-80c524d0115e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:53.917305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748cbfb-5bf1-4728-be97-5b6b5b0d2bbb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.917590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 114/232] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:53 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:53.924249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808f9b4-4672-4eaf-82fe-80c524d0115e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:53.991714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407da1a-2f04-424b-b028-e702a04204ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.992387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407da1a-2f04-424b-b028-e702a04204ec None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.992668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407da1a-2f04-424b-b028-e702a04204ec None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.992944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407da1a-2f04-424b-b028-e702a04204ec None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932/users/009f8b2c17fa408cb598aefd30ea93ff/roles/26e0da7be50c47de8620dca1c657eb4f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.995174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407da1a-2f04-424b-b028-e702a04204ec None admin] RBAC: Authorizing `identity:create_grant(project_id=b5a0684693f546a9b65d6dd77f36a932, user_id=009f8b2c17fa408cb598aefd30ea93ff, role_id=26e0da7be50c47de8620dca1c657eb4f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:53.996744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407da1a-2f04-424b-b028-e702a04204ec None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:54.026805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 119/233] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:47:53 2022] PUT /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932/users/009f8b2c17fa408cb598aefd30ea93ff/roles/26e0da7be50c47de8620dca1c657eb4f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:47:54.066654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808f9b4-4672-4eaf-82fe-80c524d0115e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:54.108592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17960cfe-30b5-43e9-8967-6b03d3a1038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:54.109529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17960cfe-30b5-43e9-8967-6b03d3a1038b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:54.110497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17960cfe-30b5-43e9-8967-6b03d3a1038b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:54.111124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17960cfe-30b5-43e9-8967-6b03d3a1038b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:54.112002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17960cfe-30b5-43e9-8967-6b03d3a1038b None admin] RBAC: Adding query filter params (role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=b5a0684693f546a9b65d6dd77f36a932, user.id=90556558521e4a8aab42c948f0552637) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:54.112446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17960cfe-30b5-43e9-8967-6b03d3a1038b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:54.114047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17960cfe-30b5-43e9-8967-6b03d3a1038b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:54.125744 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 115/234] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:53 2022] GET /identity/v3/role_assignments?user.id=90556558521e4a8aab42c948f0552637&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:55.421194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cfe4b-8c2f-42c1-a04b-ed2e893d764a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:55.422191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cfe4b-8c2f-42c1-a04b-ed2e893d764a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:55.422729 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cfe4b-8c2f-42c1-a04b-ed2e893d764a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:55.424624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 120/235] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:55.431091 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1491db-f936-4ea6-bcb4-caf9a0a0a6fd None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:55.431931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1491db-f936-4ea6-bcb4-caf9a0a0a6fd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:55.432405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1491db-f936-4ea6-bcb4-caf9a0a0a6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:55.458022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1d1491db-f936-4ea6-bcb4-caf9a0a0a6fd None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:55.459563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1d1491db-f936-4ea6-bcb4-caf9a0a0a6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:55.473957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:55.544502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 116/236] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:55.812303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4048e-6d88-4228-9e03-f3267acb6848 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:55.812593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4048e-6d88-4228-9e03-f3267acb6848 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:55.812887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4048e-6d88-4228-9e03-f3267acb6848 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:55.815142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 121/237] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:55.820698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e383c70-c195-4953-ac73-9aae98e4ba78 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:55.820993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e383c70-c195-4953-ac73-9aae98e4ba78 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:55.821304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e383c70-c195-4953-ac73-9aae98e4ba78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:55.843462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9e383c70-c195-4953-ac73-9aae98e4ba78 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:55.845199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9e383c70-c195-4953-ac73-9aae98e4ba78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:55.847232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4048e-6d88-4228-9e03-f3267acb6848 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:55.928908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 117/238] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:55.930458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a948b-f47d-45ab-9e1b-0991e17f22d8 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:55.930812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a948b-f47d-45ab-9e1b-0991e17f22d8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:55.931093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a948b-f47d-45ab-9e1b-0991e17f22d8 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:55.936275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 118/239] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:55.938031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554db8d-6e09-4af6-9948-05bf358a7c78 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:55.938890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554db8d-6e09-4af6-9948-05bf358a7c78 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:55.939344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554db8d-6e09-4af6-9948-05bf358a7c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:55.959601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4554db8d-6e09-4af6-9948-05bf358a7c78 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:55.960802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4554db8d-6e09-4af6-9948-05bf358a7c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:55.966216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4048e-6d88-4228-9e03-f3267acb6848 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:55.968026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:56.004169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3af3f-8428-43aa-8cc5-70e3d92e8914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:56.005270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3af3f-8428-43aa-8cc5-70e3d92e8914 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.005750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3af3f-8428-43aa-8cc5-70e3d92e8914 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.006226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3af3f-8428-43aa-8cc5-70e3d92e8914 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.034255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3af3f-8428-43aa-8cc5-70e3d92e8914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:56.045098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 119/240] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-85f3af3f-8428-43aa-8cc5-70e3d92e8914 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:56.045990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:56.046837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:47:56.048155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:56.050588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 122/241] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:56.056591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554db8d-6e09-4af6-9948-05bf358a7c78 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:56.060183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:56.190794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554db8d-6e09-4af6-9948-05bf358a7c78 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:56.226870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30d0f8-86c3-4ab0-9b4c-eee1d1385e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:56.227628 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d0f8-86c3-4ab0-9b4c-eee1d1385e1b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.227919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d0f8-86c3-4ab0-9b4c-eee1d1385e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.228196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d0f8-86c3-4ab0-9b4c-eee1d1385e1b None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.228873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30d0f8-86c3-4ab0-9b4c-eee1d1385e1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:56.229208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30d0f8-86c3-4ab0-9b4c-eee1d1385e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:56.230831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30d0f8-86c3-4ab0-9b4c-eee1d1385e1b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:56.241200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 120/242] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:56.246763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3af3f-8428-43aa-8cc5-70e3d92e8914 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:56.248428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30d0f8-86c3-4ab0-9b4c-eee1d1385e1b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:56.333473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:56.360599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30d0f8-86c3-4ab0-9b4c-eee1d1385e1b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:56.405459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d10ea-ae26-4ff3-b3e4-b2e186b2a3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:56.406327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d10ea-ae26-4ff3-b3e4-b2e186b2a3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.407625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d10ea-ae26-4ff3-b3e4-b2e186b2a3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.408100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d10ea-ae26-4ff3-b3e4-b2e186b2a3e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.409461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d10ea-ae26-4ff3-b3e4-b2e186b2a3e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:56.411028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d10ea-ae26-4ff3-b3e4-b2e186b2a3e9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:56.413043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 121/243] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:56.418520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d10ea-ae26-4ff3-b3e4-b2e186b2a3e9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:56.420321 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3af3f-8428-43aa-8cc5-70e3d92e8914 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:56.493935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd0344-6d82-4e32-9402-5cac0fc054a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:56.495222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd0344-6d82-4e32-9402-5cac0fc054a6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.495661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd0344-6d82-4e32-9402-5cac0fc054a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.496068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd0344-6d82-4e32-9402-5cac0fc054a6 None admin] PATH_INFO: `/v3/users/ace213c5f27745f2b31f647551104cc0` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.511958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd0344-6d82-4e32-9402-5cac0fc054a6 None admin] RBAC: Authorizing `identity:get_user(user_id=ace213c5f27745f2b31f647551104cc0)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:56.513760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd0344-6d82-4e32-9402-5cac0fc054a6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:56.515597 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 123/244] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:56 2022] GET /identity/v3/users/ace213c5f27745f2b31f647551104cc0 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:56.517661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd0344-6d82-4e32-9402-5cac0fc054a6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:56.527043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d10ea-ae26-4ff3-b3e4-b2e186b2a3e9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:56.546016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:56.572600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc297f0-990f-490c-ab85-c63c1513485f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:56.573656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc297f0-990f-490c-ab85-c63c1513485f None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.574075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc297f0-990f-490c-ab85-c63c1513485f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.574511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc297f0-990f-490c-ab85-c63c1513485f None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.575395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc297f0-990f-490c-ab85-c63c1513485f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:56.576965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc297f0-990f-490c-ab85-c63c1513485f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:56.593436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:56.608983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 122/245] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:47:56 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:47:56.615611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc297f0-990f-490c-ab85-c63c1513485f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:56.705947 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd0344-6d82-4e32-9402-5cac0fc054a6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:56.757160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced851b1-0dbc-4a12-bf4c-1ca8b27e757c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:56.758053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced851b1-0dbc-4a12-bf4c-1ca8b27e757c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.758499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced851b1-0dbc-4a12-bf4c-1ca8b27e757c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.758927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced851b1-0dbc-4a12-bf4c-1ca8b27e757c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.774423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced851b1-0dbc-4a12-bf4c-1ca8b27e757c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ced851b1-0dbc-4a12-bf4c-1ca8b27e757c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:56.779490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:56.780473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:56.781279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 124/246] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:47:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:56.782474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf456ef-8028-45f5-b2d0-2f679fe158ae None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.782772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf456ef-8028-45f5-b2d0-2f679fe158ae None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.783460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf456ef-8028-45f5-b2d0-2f679fe158ae None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.784927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 125/247] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:56.786015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a369e34-c8fe-4bf4-958c-24f851be6f83 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.786299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a369e34-c8fe-4bf4-958c-24f851be6f83 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.786802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a369e34-c8fe-4bf4-958c-24f851be6f83 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.789170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 126/248] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:56.790031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a369e34-c8fe-4bf4-958c-24f851be6f83 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:56.820964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc297f0-990f-490c-ab85-c63c1513485f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:56.879372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444edf7-e4d0-44b1-a01a-c07344a4b3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:56.879995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444edf7-e4d0-44b1-a01a-c07344a4b3bb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.880935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444edf7-e4d0-44b1-a01a-c07344a4b3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.880935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444edf7-e4d0-44b1-a01a-c07344a4b3bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.943194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:56.965046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:56.980128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444edf7-e4d0-44b1-a01a-c07344a4b3bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:56.983074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444edf7-e4d0-44b1-a01a-c07344a4b3bb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:56.993796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a369e34-c8fe-4bf4-958c-24f851be6f83 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:57.039060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc4586-7c73-4791-8d35-4d97b52dfb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:57.039590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc4586-7c73-4791-8d35-4d97b52dfb4d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.039868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc4586-7c73-4791-8d35-4d97b52dfb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.040142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc4586-7c73-4791-8d35-4d97b52dfb4d None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.040808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc4586-7c73-4791-8d35-4d97b52dfb4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:57.041228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc4586-7c73-4791-8d35-4d97b52dfb4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:57.042611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc4586-7c73-4791-8d35-4d97b52dfb4d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:57.053373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 127/249] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:57.054613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3ed16-d963-4f82-8017-fcef0a3465b2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.054896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3ed16-d963-4f82-8017-fcef0a3465b2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.055169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3ed16-d963-4f82-8017-fcef0a3465b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.079568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9da3ed16-d963-4f82-8017-fcef0a3465b2 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:57.081168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9da3ed16-d963-4f82-8017-fcef0a3465b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:57.158013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 123/250] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:57.159664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67f979-210b-4f8b-956d-cfc11db1527e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.160189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67f979-210b-4f8b-956d-cfc11db1527e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.160766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67f979-210b-4f8b-956d-cfc11db1527e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.162039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 128/251] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:57.162894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3ed16-d963-4f82-8017-fcef0a3465b2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:57.181413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9f67f979-210b-4f8b-956d-cfc11db1527e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:57.183025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9f67f979-210b-4f8b-956d-cfc11db1527e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:57.237245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:57.261701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 124/252] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:57.262807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67f979-210b-4f8b-956d-cfc11db1527e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:57.280470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3ed16-d963-4f82-8017-fcef0a3465b2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:57.289586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:57.317405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988556bb-4dfd-4f50-b143-945b3b44bb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:57.318044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988556bb-4dfd-4f50-b143-945b3b44bb7f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.318563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988556bb-4dfd-4f50-b143-945b3b44bb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.318563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988556bb-4dfd-4f50-b143-945b3b44bb7f None admin] PATH_INFO: `/v3/users/009f8b2c17fa408cb598aefd30ea93ff` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.319982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988556bb-4dfd-4f50-b143-945b3b44bb7f None admin] RBAC: Authorizing `identity:get_user(user_id=009f8b2c17fa408cb598aefd30ea93ff)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:57.321463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988556bb-4dfd-4f50-b143-945b3b44bb7f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:57.323051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 129/253] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:57 2022] GET /identity/v3/users/009f8b2c17fa408cb598aefd30ea93ff => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:57.328637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988556bb-4dfd-4f50-b143-945b3b44bb7f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:57.372441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67f979-210b-4f8b-956d-cfc11db1527e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:57.460624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c80f5-bf1b-4aa3-8136-c5b5c9784750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:57.462822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c80f5-bf1b-4aa3-8136-c5b5c9784750 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c80f5-bf1b-4aa3-8136-c5b5c9784750 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.463785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c80f5-bf1b-4aa3-8136-c5b5c9784750 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.465899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c80f5-bf1b-4aa3-8136-c5b5c9784750 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:57.467628 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c80f5-bf1b-4aa3-8136-c5b5c9784750 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:57.481938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 125/254] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:57 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:57.487876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c80f5-bf1b-4aa3-8136-c5b5c9784750 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:57.544907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988556bb-4dfd-4f50-b143-945b3b44bb7f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:57.610785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7eb0b-25a8-4059-acfb-0626973f673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:57.611395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7eb0b-25a8-4059-acfb-0626973f673d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.611676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7eb0b-25a8-4059-acfb-0626973f673d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.611950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7eb0b-25a8-4059-acfb-0626973f673d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.629409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:57.653487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:57.671974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c80f5-bf1b-4aa3-8136-c5b5c9784750 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:57.708973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7eb0b-25a8-4059-acfb-0626973f673d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:57.710492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7eb0b-25a8-4059-acfb-0626973f673d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:57.715230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee67f5b-1d4a-49b7-800f-2d3619575a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:57.716054 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee67f5b-1d4a-49b7-800f-2d3619575a98 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.716518 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee67f5b-1d4a-49b7-800f-2d3619575a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.716968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee67f5b-1d4a-49b7-800f-2d3619575a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.717901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee67f5b-1d4a-49b7-800f-2d3619575a98 None admin] RBAC: Adding query filter params (role.id=69d0a4169025430084668f95838669b1, scope.project.id=b5a0684693f546a9b65d6dd77f36a932, user.id=ace213c5f27745f2b31f647551104cc0) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:57.718416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee67f5b-1d4a-49b7-800f-2d3619575a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:57.721387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee67f5b-1d4a-49b7-800f-2d3619575a98 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:57.729683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 126/255] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:57 2022] GET /identity/v3/role_assignments?user.id=ace213c5f27745f2b31f647551104cc0&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=69d0a4169025430084668f95838669b1 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:47:57.731090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee67f5b-1d4a-49b7-800f-2d3619575a98 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:57.898260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 130/256] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:47:57.899132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7eb0b-25a8-4059-acfb-0626973f673d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:57.917268 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee67f5b-1d4a-49b7-800f-2d3619575a98 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:57.957852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073c949-11ae-44dd-8ea8-d25b76d73c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:57.958779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073c949-11ae-44dd-8ea8-d25b76d73c78 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.959239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073c949-11ae-44dd-8ea8-d25b76d73c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.959721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073c949-11ae-44dd-8ea8-d25b76d73c78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.972851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073c949-11ae-44dd-8ea8-d25b76d73c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c073c949-11ae-44dd-8ea8-d25b76d73c78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:57.980696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:57.981695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:57.982482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:57.985053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 127/257] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:57.986304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073c949-11ae-44dd-8ea8-d25b76d73c78 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:57.988204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:58.008962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:58.102540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7eb0b-25a8-4059-acfb-0626973f673d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.148696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8d632-58ef-416c-af31-7c64252a7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.149667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8d632-58ef-416c-af31-7c64252a7bdf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.150166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8d632-58ef-416c-af31-7c64252a7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.150616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8d632-58ef-416c-af31-7c64252a7bdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:58.162455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8d632-58ef-416c-af31-7c64252a7bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:58.165212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073c949-11ae-44dd-8ea8-d25b76d73c78 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c9f8d632-58ef-416c-af31-7c64252a7bdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:58.167880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:58.169036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:47:58.169871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:47:58.171768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 131/258] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:47:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:58.172911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8d632-58ef-416c-af31-7c64252a7bdf None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:58.230966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46596e9-5176-41d0-9f51-dbe15ada8920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.233003 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46596e9-5176-41d0-9f51-dbe15ada8920 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.233583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46596e9-5176-41d0-9f51-dbe15ada8920 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.234034 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46596e9-5176-41d0-9f51-dbe15ada8920 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:58.236395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46596e9-5176-41d0-9f51-dbe15ada8920 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:58.239302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46596e9-5176-41d0-9f51-dbe15ada8920 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:58.243174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 128/259] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:57 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:58.247186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46596e9-5176-41d0-9f51-dbe15ada8920 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:58.331225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8d632-58ef-416c-af31-7c64252a7bdf None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.333293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:58.394921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c2175-72c7-45e3-b9ed-c192b0b5f89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.395535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c2175-72c7-45e3-b9ed-c192b0b5f89a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.395814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c2175-72c7-45e3-b9ed-c192b0b5f89a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.396089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c2175-72c7-45e3-b9ed-c192b0b5f89a None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:58.396765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c2175-72c7-45e3-b9ed-c192b0b5f89a None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:58.398475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c2175-72c7-45e3-b9ed-c192b0b5f89a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:58.400683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c2175-72c7-45e3-b9ed-c192b0b5f89a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:58.415960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 132/260] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:58.416825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c2175-72c7-45e3-b9ed-c192b0b5f89a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:58.442067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:58.483288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46596e9-5176-41d0-9f51-dbe15ada8920 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.536475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c2175-72c7-45e3-b9ed-c192b0b5f89a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.547037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbadb77-cdbe-4b57-b4f8-2b857f6f1696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.548587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbadb77-cdbe-4b57-b4f8-2b857f6f1696 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.549089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbadb77-cdbe-4b57-b4f8-2b857f6f1696 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.549559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbadb77-cdbe-4b57-b4f8-2b857f6f1696 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:58.550522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbadb77-cdbe-4b57-b4f8-2b857f6f1696 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:58.551136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbadb77-cdbe-4b57-b4f8-2b857f6f1696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:58.552685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbadb77-cdbe-4b57-b4f8-2b857f6f1696 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:58.567711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 129/261] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:47:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:58.568994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbadb77-cdbe-4b57-b4f8-2b857f6f1696 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:58.578373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9a8fc-d04e-49bf-9ef6-a03f17e1115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.579651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9a8fc-d04e-49bf-9ef6-a03f17e1115b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.579905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9a8fc-d04e-49bf-9ef6-a03f17e1115b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.580180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9a8fc-d04e-49bf-9ef6-a03f17e1115b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:58.580917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9a8fc-d04e-49bf-9ef6-a03f17e1115b None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=b5a0684693f546a9b65d6dd77f36a932, user.id=009f8b2c17fa408cb598aefd30ea93ff) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:58.581264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9a8fc-d04e-49bf-9ef6-a03f17e1115b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:58.582721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9a8fc-d04e-49bf-9ef6-a03f17e1115b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:58.589008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 133/262] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:58 2022] GET /identity/v3/role_assignments?user.id=009f8b2c17fa408cb598aefd30ea93ff&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:58.589840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9a8fc-d04e-49bf-9ef6-a03f17e1115b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:58.620003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:58.700181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9a8fc-d04e-49bf-9ef6-a03f17e1115b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.737949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e3231-7278-4a70-b292-c4d3e0967dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.738474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e3231-7278-4a70-b292-c4d3e0967dfc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.738758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e3231-7278-4a70-b292-c4d3e0967dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.739033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e3231-7278-4a70-b292-c4d3e0967dfc None admin] PATH_INFO: `/v3/users/009f8b2c17fa408cb598aefd30ea93ff` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:58.740191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e3231-7278-4a70-b292-c4d3e0967dfc None admin] RBAC: Authorizing `identity:get_user(user_id=009f8b2c17fa408cb598aefd30ea93ff)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:58.741744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e3231-7278-4a70-b292-c4d3e0967dfc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:58.743624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 134/263] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:58 2022] GET /identity/v3/users/009f8b2c17fa408cb598aefd30ea93ff => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:58.749643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e3231-7278-4a70-b292-c4d3e0967dfc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:58.757389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbadb77-cdbe-4b57-b4f8-2b857f6f1696 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.802168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbe4e7-759c-4a69-8b1c-07985b87946a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.803176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbe4e7-759c-4a69-8b1c-07985b87946a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.803720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbe4e7-759c-4a69-8b1c-07985b87946a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.804321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbe4e7-759c-4a69-8b1c-07985b87946a None admin] PATH_INFO: `/v3/users/90556558521e4a8aab42c948f0552637` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:58.805699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbe4e7-759c-4a69-8b1c-07985b87946a None admin] RBAC: Authorizing `identity:get_user(user_id=90556558521e4a8aab42c948f0552637)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:58.807564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbe4e7-759c-4a69-8b1c-07985b87946a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:58.809413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 130/264] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:47:58 2022] GET /identity/v3/users/90556558521e4a8aab42c948f0552637 => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:58.818495 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbe4e7-759c-4a69-8b1c-07985b87946a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:58.845468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:58.895394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:58.908261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e3231-7278-4a70-b292-c4d3e0967dfc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.981609 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f646c-3f86-4f5f-862e-8305a678e9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.982147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f646c-3f86-4f5f-862e-8305a678e9cc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.982431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f646c-3f86-4f5f-862e-8305a678e9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.982704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f646c-3f86-4f5f-862e-8305a678e9cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.028332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbe4e7-759c-4a69-8b1c-07985b87946a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:59.078537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f646c-3f86-4f5f-862e-8305a678e9cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:59.078765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dda2f1-6e24-4f9e-9e1d-e7225a593535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:59.079382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dda2f1-6e24-4f9e-9e1d-e7225a593535 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.079730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dda2f1-6e24-4f9e-9e1d-e7225a593535 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.079937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dda2f1-6e24-4f9e-9e1d-e7225a593535 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.080964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f646c-3f86-4f5f-862e-8305a678e9cc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:59.081069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dda2f1-6e24-4f9e-9e1d-e7225a593535 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:59.082546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dda2f1-6e24-4f9e-9e1d-e7225a593535 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:59.085347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 131/265] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:58 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:59.086557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565d1e2-b522-42bd-a697-ad9538a4ec85 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.086841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565d1e2-b522-42bd-a697-ad9538a4ec85 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.087110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565d1e2-b522-42bd-a697-ad9538a4ec85 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.090167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 132/266] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:59.091070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565d1e2-b522-42bd-a697-ad9538a4ec85 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:59.203355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:59.206489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:59.213294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 135/267] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:47:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:47:59.220324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1ad64-bc12-482e-8a61-fb011fb78db3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.220324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1ad64-bc12-482e-8a61-fb011fb78db3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.220324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1ad64-bc12-482e-8a61-fb011fb78db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.241910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-03f1ad64-bc12-482e-8a61-fb011fb78db3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:59.243045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-03f1ad64-bc12-482e-8a61-fb011fb78db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:59.248341 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565d1e2-b522-42bd-a697-ad9538a4ec85 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:59.291633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc0e2c-01b2-4176-9acd-5ee3777095c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:59.292656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc0e2c-01b2-4176-9acd-5ee3777095c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.293852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc0e2c-01b2-4176-9acd-5ee3777095c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.294374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc0e2c-01b2-4176-9acd-5ee3777095c6 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932/users/90556558521e4a8aab42c948f0552637/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.296680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc0e2c-01b2-4176-9acd-5ee3777095c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5a0684693f546a9b65d6dd77f36a932, user_id=90556558521e4a8aab42c948f0552637, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:59.299509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc0e2c-01b2-4176-9acd-5ee3777095c6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:59.318769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 133/268] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:47:59 2022] PUT /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932/users/90556558521e4a8aab42c948f0552637/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:47:59.319757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc0e2c-01b2-4176-9acd-5ee3777095c6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:59.333431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 136/269] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:59.334830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c6736-9c45-438d-8ea3-1d133fbe9d57 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.335127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c6736-9c45-438d-8ea3-1d133fbe9d57 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.335407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c6736-9c45-438d-8ea3-1d133fbe9d57 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.337861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 137/270] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:59.345237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38af3c4-a61f-42a7-8201-91bf3750f260 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.345701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38af3c4-a61f-42a7-8201-91bf3750f260 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.346046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38af3c4-a61f-42a7-8201-91bf3750f260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.371213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c38af3c4-a61f-42a7-8201-91bf3750f260 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:59.372582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c38af3c4-a61f-42a7-8201-91bf3750f260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:59.457321 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 138/271] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:47:59.460724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc0e2c-01b2-4176-9acd-5ee3777095c6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:59.497859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05193c-622b-4e5c-bd96-6212b12ee044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:59.498594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05193c-622b-4e5c-bd96-6212b12ee044 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.498875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05193c-622b-4e5c-bd96-6212b12ee044 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.499151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05193c-622b-4e5c-bd96-6212b12ee044 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.501383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05193c-622b-4e5c-bd96-6212b12ee044 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:59.502832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05193c-622b-4e5c-bd96-6212b12ee044 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:59.504827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 134/272] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:47:59 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:59.510612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38af3c4-a61f-42a7-8201-91bf3750f260 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:59.577239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:59.625420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38af3c4-a61f-42a7-8201-91bf3750f260 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:59.668659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6a399-f6a3-4d83-a28e-1be1b6fb770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:59.669590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6a399-f6a3-4d83-a28e-1be1b6fb770f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.670050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6a399-f6a3-4d83-a28e-1be1b6fb770f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.670527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6a399-f6a3-4d83-a28e-1be1b6fb770f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.671428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6a399-f6a3-4d83-a28e-1be1b6fb770f None admin] RBAC: Adding query filter params (role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=b5a0684693f546a9b65d6dd77f36a932, user.id=009f8b2c17fa408cb598aefd30ea93ff) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:47:59.671922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6a399-f6a3-4d83-a28e-1be1b6fb770f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:59.674096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6a399-f6a3-4d83-a28e-1be1b6fb770f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:59.680785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 139/273] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:47:59 2022] GET /identity/v3/role_assignments?user.id=009f8b2c17fa408cb598aefd30ea93ff&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:59.687623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05193c-622b-4e5c-bd96-6212b12ee044 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:59.699409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:59.831819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05193c-622b-4e5c-bd96-6212b12ee044 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:59.835336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6a399-f6a3-4d83-a28e-1be1b6fb770f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:59.869885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c43ad1-e60e-4280-85e8-b1d4200522a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:59.870522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c43ad1-e60e-4280-85e8-b1d4200522a7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.870796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c43ad1-e60e-4280-85e8-b1d4200522a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.871232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c43ad1-e60e-4280-85e8-b1d4200522a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.883328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c43ad1-e60e-4280-85e8-b1d4200522a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c6c43ad1-e60e-4280-85e8-b1d4200522a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:47:59.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:47:59.889607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:47:59.890388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 135/274] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:47:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:47:59.896098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c43ad1-e60e-4280-85e8-b1d4200522a7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:59.977639 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6a399-f6a3-4d83-a28e-1be1b6fb770f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:59.980087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:00.007070 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:00.031390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38320553-2b3d-4f52-8d72-be37b0e3abc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:00.032114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38320553-2b3d-4f52-8d72-be37b0e3abc6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.032586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38320553-2b3d-4f52-8d72-be37b0e3abc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.033047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38320553-2b3d-4f52-8d72-be37b0e3abc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.035404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38320553-2b3d-4f52-8d72-be37b0e3abc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:00.036875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38320553-2b3d-4f52-8d72-be37b0e3abc6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:00.038498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 140/275] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:47:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:00.041391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c43ad1-e60e-4280-85e8-b1d4200522a7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:00.043957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38320553-2b3d-4f52-8d72-be37b0e3abc6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:00.088729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4443db3-3423-4594-9b2a-9289ebd3074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:00.089760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4443db3-3423-4594-9b2a-9289ebd3074e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.090245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4443db3-3423-4594-9b2a-9289ebd3074e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.090716 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4443db3-3423-4594-9b2a-9289ebd3074e None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.091600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4443db3-3423-4594-9b2a-9289ebd3074e None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:00.092762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4443db3-3423-4594-9b2a-9289ebd3074e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:00.095662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4443db3-3423-4594-9b2a-9289ebd3074e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:00.110986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 136/276] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:47:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:00.116435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4443db3-3423-4594-9b2a-9289ebd3074e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:00.180968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38320553-2b3d-4f52-8d72-be37b0e3abc6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:00.220181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c7f06-296d-4d87-9e68-c266963ad6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:00.221445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c7f06-296d-4d87-9e68-c266963ad6e8 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.221995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c7f06-296d-4d87-9e68-c266963ad6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.222467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c7f06-296d-4d87-9e68-c266963ad6e8 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.223344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c7f06-296d-4d87-9e68-c266963ad6e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:00.225115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c7f06-296d-4d87-9e68-c266963ad6e8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:00.236781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4443db3-3423-4594-9b2a-9289ebd3074e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:00.252912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 141/277] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:00 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:48:00.287915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:00.290054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264fb91-4ffd-4959-92a2-113fc95fc655 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.290603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264fb91-4ffd-4959-92a2-113fc95fc655 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.291088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264fb91-4ffd-4959-92a2-113fc95fc655 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.293144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 142/278] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:00.298082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e149a2a-d58b-4a43-aa80-1027361512ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:00.298762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146893fb-0284-424f-bd91-212ff73867ee None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.299172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149a2a-d58b-4a43-aa80-1027361512ba None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.299488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146893fb-0284-424f-bd91-212ff73867ee None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.299591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149a2a-d58b-4a43-aa80-1027361512ba None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.299761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149a2a-d58b-4a43-aa80-1027361512ba None admin] PATH_INFO: `/v3/users/e23fa84869b64ac798c7961d538fba97` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.300218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146893fb-0284-424f-bd91-212ff73867ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.317179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e149a2a-d58b-4a43-aa80-1027361512ba None admin] RBAC: Authorizing `identity:get_user(user_id=e23fa84869b64ac798c7961d538fba97)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:00.318686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e149a2a-d58b-4a43-aa80-1027361512ba None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:00.320900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 137/279] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:00 2022] GET /identity/v3/users/e23fa84869b64ac798c7961d538fba97 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:00.323631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-146893fb-0284-424f-bd91-212ff73867ee None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:00.325180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-146893fb-0284-424f-bd91-212ff73867ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:00.326835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e149a2a-d58b-4a43-aa80-1027361512ba None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:00.371942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:00.415300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 143/280] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:00.450960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e149a2a-d58b-4a43-aa80-1027361512ba None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:00.488729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c980d-815d-4ab9-8c6f-85c88df29eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:00.489710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c980d-815d-4ab9-8c6f-85c88df29eec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.490214 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c980d-815d-4ab9-8c6f-85c88df29eec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.490685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c980d-815d-4ab9-8c6f-85c88df29eec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.491760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c980d-815d-4ab9-8c6f-85c88df29eec None admin] RBAC: Adding query filter params (role.id=e407aec03c22425591fa0e52a40edbc1, user.id=e23fa84869b64ac798c7961d538fba97, scope.system=all) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:00.492278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c980d-815d-4ab9-8c6f-85c88df29eec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:00.493982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c980d-815d-4ab9-8c6f-85c88df29eec None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:00.500163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 138/281] 10.222.0.78 () {56 vars in 1308 bytes} [Tue Aug 30 00:48:00 2022] GET /identity/v3/role_assignments?user.id=e23fa84869b64ac798c7961d538fba97&scope.system=all&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:00.695106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146893fb-0284-424f-bd91-212ff73867ee None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:00.810323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146893fb-0284-424f-bd91-212ff73867ee None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:00.850450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da1bb1-0047-437d-8586-071a13971389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:00.852105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da1bb1-0047-437d-8586-071a13971389 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.852594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da1bb1-0047-437d-8586-071a13971389 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.853068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da1bb1-0047-437d-8586-071a13971389 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.865472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da1bb1-0047-437d-8586-071a13971389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-53da1bb1-0047-437d-8586-071a13971389 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:00.870812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:00.871689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:00.872473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:00.875061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 144/282] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:00.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c980d-815d-4ab9-8c6f-85c88df29eec None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:00.957452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:01.023826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c980d-815d-4ab9-8c6f-85c88df29eec None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:01.069941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64c811-0edc-4502-ae56-3c7bfa9f7aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:01.070995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64c811-0edc-4502-ae56-3c7bfa9f7aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:01.071424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64c811-0edc-4502-ae56-3c7bfa9f7aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:01.071704 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64c811-0edc-4502-ae56-3c7bfa9f7aa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:01.072384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64c811-0edc-4502-ae56-3c7bfa9f7aa9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:01.072687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64c811-0edc-4502-ae56-3c7bfa9f7aa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:01.074164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64c811-0edc-4502-ae56-3c7bfa9f7aa9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:01.083640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 139/283] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:01.089191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da1bb1-0047-437d-8586-071a13971389 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:01.128891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:01.216204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da1bb1-0047-437d-8586-071a13971389 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:01.259628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c6360-8c4d-4de8-92c7-665074941e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:01.260278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c6360-8c4d-4de8-92c7-665074941e01 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:01.260558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c6360-8c4d-4de8-92c7-665074941e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:01.260825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c6360-8c4d-4de8-92c7-665074941e01 None admin] PATH_INFO: `/v3/users/ace213c5f27745f2b31f647551104cc0` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:01.262018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c6360-8c4d-4de8-92c7-665074941e01 None admin] RBAC: Authorizing `identity:get_user(user_id=ace213c5f27745f2b31f647551104cc0)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:01.263475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c6360-8c4d-4de8-92c7-665074941e01 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:01.265060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 145/284] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:01 2022] GET /identity/v3/users/ace213c5f27745f2b31f647551104cc0 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:01.271439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64c811-0edc-4502-ae56-3c7bfa9f7aa9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:01.406654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64c811-0edc-4502-ae56-3c7bfa9f7aa9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:01.433514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:01.456920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a1773-bec0-47dd-97aa-f866a478c56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:01.457639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a1773-bec0-47dd-97aa-f866a478c56d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:01.465505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a1773-bec0-47dd-97aa-f866a478c56d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:01.465793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a1773-bec0-47dd-97aa-f866a478c56d None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:01.467042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a1773-bec0-47dd-97aa-f866a478c56d None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:01.468431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a1773-bec0-47dd-97aa-f866a478c56d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:01.470371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 140/285] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:01 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:01.476258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c6360-8c4d-4de8-92c7-665074941e01 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:01.597058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54663cb4-8c41-4f47-bab3-1256670c7fec None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:01.597396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54663cb4-8c41-4f47-bab3-1256670c7fec None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:01.597638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54663cb4-8c41-4f47-bab3-1256670c7fec None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:01.599132 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 141/286] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:01.601712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:01.607900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c6360-8c4d-4de8-92c7-665074941e01 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:01.618249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a4aa3-c9e6-481b-b9ff-9234cfcc141e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:01.618809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a4aa3-c9e6-481b-b9ff-9234cfcc141e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:01.619346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a4aa3-c9e6-481b-b9ff-9234cfcc141e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:01.650649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a27f5-541c-43a8-9dd2-cba26da6eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:01.651323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a27f5-541c-43a8-9dd2-cba26da6eda5 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:01.651610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a27f5-541c-43a8-9dd2-cba26da6eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:01.651886 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a27f5-541c-43a8-9dd2-cba26da6eda5 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932/users/ace213c5f27745f2b31f647551104cc0/roles/69d0a4169025430084668f95838669b1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:01.658621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2e8a4aa3-c9e6-481b-b9ff-9234cfcc141e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:01.659835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2e8a4aa3-c9e6-481b-b9ff-9234cfcc141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:01.661514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a27f5-541c-43a8-9dd2-cba26da6eda5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5a0684693f546a9b65d6dd77f36a932, user_id=ace213c5f27745f2b31f647551104cc0, role_id=69d0a4169025430084668f95838669b1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:01.662979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a27f5-541c-43a8-9dd2-cba26da6eda5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:01.710806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 146/287] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:01 2022] PUT /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932/users/ace213c5f27745f2b31f647551104cc0/roles/69d0a4169025430084668f95838669b1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:01.753503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 142/288] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:02.058868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a27f5-541c-43a8-9dd2-cba26da6eda5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:02.198440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a27f5-541c-43a8-9dd2-cba26da6eda5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:02.258634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc2d5-3586-4b17-9217-74623ebf45f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:02.258903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc2d5-3586-4b17-9217-74623ebf45f0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:02.259079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc2d5-3586-4b17-9217-74623ebf45f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:02.259330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc2d5-3586-4b17-9217-74623ebf45f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:02.282565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dc2d5-3586-4b17-9217-74623ebf45f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-0e4dc2d5-3586-4b17-9217-74623ebf45f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:02.287897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:02.288751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:02.289664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 147/289] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:02.294947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a4aa3-c9e6-481b-b9ff-9234cfcc141e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:02.425057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a4aa3-c9e6-481b-b9ff-9234cfcc141e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:02.445151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:02.471486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d93b6-d61b-48e2-a767-13a7db204ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:02.472143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d93b6-d61b-48e2-a767-13a7db204ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:02.472423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d93b6-d61b-48e2-a767-13a7db204ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:02.472689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d93b6-d61b-48e2-a767-13a7db204ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:02.473386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d93b6-d61b-48e2-a767-13a7db204ab6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:02.473680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d93b6-d61b-48e2-a767-13a7db204ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:02.475025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d93b6-d61b-48e2-a767-13a7db204ab6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:02.483894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 143/290] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:02.490090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc2d5-3586-4b17-9217-74623ebf45f0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:02.527843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:02.665114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc2d5-3586-4b17-9217-74623ebf45f0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:02.718035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336440f-657a-42e6-9554-dfca980f5f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:02.718783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336440f-657a-42e6-9554-dfca980f5f94 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:02.718940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336440f-657a-42e6-9554-dfca980f5f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:02.719090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336440f-657a-42e6-9554-dfca980f5f94 None admin] PATH_INFO: `/v3/users/90556558521e4a8aab42c948f0552637` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:02.725796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336440f-657a-42e6-9554-dfca980f5f94 None admin] RBAC: Authorizing `identity:get_user(user_id=90556558521e4a8aab42c948f0552637)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:02.727315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336440f-657a-42e6-9554-dfca980f5f94 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:02.729254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 148/291] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:02 2022] GET /identity/v3/users/90556558521e4a8aab42c948f0552637 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:02.734949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d93b6-d61b-48e2-a767-13a7db204ab6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:02.854606 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407264c-7986-4f9f-bcce-e5537fae382f None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:02.854966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407264c-7986-4f9f-bcce-e5537fae382f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:02.855154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407264c-7986-4f9f-bcce-e5537fae382f None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:02.856762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 149/292] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:02.872445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2253df4-eced-448c-962b-85e72166dc89 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:02.872730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2253df4-eced-448c-962b-85e72166dc89 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:02.872987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2253df4-eced-448c-962b-85e72166dc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:02.891963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d93b6-d61b-48e2-a767-13a7db204ab6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:02.913114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b2253df4-eced-448c-962b-85e72166dc89 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:02.914314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b2253df4-eced-448c-962b-85e72166dc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:02.936589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666055a2-e638-4238-9378-c2c3d3a60686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:02.939142 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666055a2-e638-4238-9378-c2c3d3a60686 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:02.939439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666055a2-e638-4238-9378-c2c3d3a60686 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:02.939714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666055a2-e638-4238-9378-c2c3d3a60686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:02.999439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.021635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 150/293] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:03.022887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b82ee-0987-45df-9a77-dae1e53f5cc1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.023166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b82ee-0987-45df-9a77-dae1e53f5cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.023428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b82ee-0987-45df-9a77-dae1e53f5cc1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.024819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 151/294] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:03.025651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8f8b1-b19f-4589-b4f9-646d3745f78d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.025924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8f8b1-b19f-4589-b4f9-646d3745f78d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.026190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8f8b1-b19f-4589-b4f9-646d3745f78d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.027499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 152/295] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:03.029736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaf878-bbdf-4012-933c-bced0fcb7462 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.030008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaf878-bbdf-4012-933c-bced0fcb7462 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.030272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaf878-bbdf-4012-933c-bced0fcb7462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.042864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666055a2-e638-4238-9378-c2c3d3a60686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:03.044907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666055a2-e638-4238-9378-c2c3d3a60686 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:03.049328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-dfdaf878-bbdf-4012-933c-bced0fcb7462 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:03.049866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-dfdaf878-bbdf-4012-933c-bced0fcb7462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:03.119718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 153/296] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:03.120900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d13148-2325-4b1e-a51f-b3bc3223294b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.121214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d13148-2325-4b1e-a51f-b3bc3223294b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.121466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d13148-2325-4b1e-a51f-b3bc3223294b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.129520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.140217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-84d13148-2325-4b1e-a51f-b3bc3223294b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:03.141937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-84d13148-2325-4b1e-a51f-b3bc3223294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:03.177889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 144/297] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:03.183572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666055a2-e638-4238-9378-c2c3d3a60686 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.227951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 154/298] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:03.277487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.310747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666055a2-e638-4238-9378-c2c3d3a60686 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:03.348632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62ea59-7917-4f02-8f5f-d2893aeb2b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:03.349285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62ea59-7917-4f02-8f5f-d2893aeb2b5a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.349563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62ea59-7917-4f02-8f5f-d2893aeb2b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.349829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62ea59-7917-4f02-8f5f-d2893aeb2b5a None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.350951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62ea59-7917-4f02-8f5f-d2893aeb2b5a None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:03.352532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62ea59-7917-4f02-8f5f-d2893aeb2b5a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:03.354657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 145/299] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:03 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:03.360973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d13148-2325-4b1e-a51f-b3bc3223294b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.376351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62ea59-7917-4f02-8f5f-d2893aeb2b5a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.484997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.516300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d13148-2325-4b1e-a51f-b3bc3223294b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:03.544867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62ea59-7917-4f02-8f5f-d2893aeb2b5a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:03.565940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f0200-2b9d-40bf-9785-af1558fbc7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:03.567156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f0200-2b9d-40bf-9785-af1558fbc7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.568210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f0200-2b9d-40bf-9785-af1558fbc7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.569952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f0200-2b9d-40bf-9785-af1558fbc7f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.570897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f0200-2b9d-40bf-9785-af1558fbc7f5 None admin] RBAC: Adding query filter params (role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=b5a0684693f546a9b65d6dd77f36a932, user.id=90556558521e4a8aab42c948f0552637) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:03.571408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f0200-2b9d-40bf-9785-af1558fbc7f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:03.573905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f0200-2b9d-40bf-9785-af1558fbc7f5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:03.585416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 155/300] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:03 2022] GET /identity/v3/role_assignments?user.id=90556558521e4a8aab42c948f0552637&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:03.586683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f0200-2b9d-40bf-9785-af1558fbc7f5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.588809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f271e0-9c52-4fe4-b3d0-a4c3bc8dc167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:03.589591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f271e0-9c52-4fe4-b3d0-a4c3bc8dc167 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.590210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f271e0-9c52-4fe4-b3d0-a4c3bc8dc167 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.590590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f271e0-9c52-4fe4-b3d0-a4c3bc8dc167 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.603456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.607866 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f271e0-9c52-4fe4-b3d0-a4c3bc8dc167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a8f271e0-9c52-4fe4-b3d0-a4c3bc8dc167 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:03.615511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:03.616706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:03.617689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:03.620259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 146/301] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:03.626115 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f271e0-9c52-4fe4-b3d0-a4c3bc8dc167 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.729910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f0200-2b9d-40bf-9785-af1558fbc7f5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:03.773650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf4cf3-ded7-4a7d-b981-c14787881374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:03.773834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f271e0-9c52-4fe4-b3d0-a4c3bc8dc167 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:03.774654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf4cf3-ded7-4a7d-b981-c14787881374 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.775129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf4cf3-ded7-4a7d-b981-c14787881374 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.775409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf4cf3-ded7-4a7d-b981-c14787881374 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.791028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf4cf3-ded7-4a7d-b981-c14787881374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-27bf4cf3-ded7-4a7d-b981-c14787881374 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:03.796647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:03.798148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:03.799061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:03.802797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 156/302] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:03.803917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf4cf3-ded7-4a7d-b981-c14787881374 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.807176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.824705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230268-3296-4ebb-9626-2fe2f93e6893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:03.825375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230268-3296-4ebb-9626-2fe2f93e6893 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.825652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230268-3296-4ebb-9626-2fe2f93e6893 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.825922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230268-3296-4ebb-9626-2fe2f93e6893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.827048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f230268-3296-4ebb-9626-2fe2f93e6893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:03.828472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f230268-3296-4ebb-9626-2fe2f93e6893 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:03.831463 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 147/303] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:03.832567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230268-3296-4ebb-9626-2fe2f93e6893 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.914433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.920864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf4cf3-ded7-4a7d-b981-c14787881374 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:03.957605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230268-3296-4ebb-9626-2fe2f93e6893 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:03.958497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f147eee-5da2-4849-8fcd-3fec6e8bd0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:03.959226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f147eee-5da2-4849-8fcd-3fec6e8bd0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.959509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f147eee-5da2-4849-8fcd-3fec6e8bd0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.959780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f147eee-5da2-4849-8fcd-3fec6e8bd0c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.960448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f147eee-5da2-4849-8fcd-3fec6e8bd0c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:03.960751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f147eee-5da2-4849-8fcd-3fec6e8bd0c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:03.962176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f147eee-5da2-4849-8fcd-3fec6e8bd0c7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:03.971584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 157/304] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:03.972789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f147eee-5da2-4849-8fcd-3fec6e8bd0c7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.995108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bf5d3-e180-44c4-96f4-0c8d39ede4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:03.995711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bf5d3-e180-44c4-96f4-0c8d39ede4ec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.995986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bf5d3-e180-44c4-96f4-0c8d39ede4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.996253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bf5d3-e180-44c4-96f4-0c8d39ede4ec None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.996916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bf5d3-e180-44c4-96f4-0c8d39ede4ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:03.997243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bf5d3-e180-44c4-96f4-0c8d39ede4ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:03.998580 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bf5d3-e180-44c4-96f4-0c8d39ede4ec None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.007514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 148/305] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:04.012716 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bf5d3-e180-44c4-96f4-0c8d39ede4ec None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.047301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.090422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f147eee-5da2-4849-8fcd-3fec6e8bd0c7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.119527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bf5d3-e180-44c4-96f4-0c8d39ede4ec None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.126041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca372a-e4f1-4e10-9741-fcd59d75cb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.126647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca372a-e4f1-4e10-9741-fcd59d75cb73 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.126921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca372a-e4f1-4e10-9741-fcd59d75cb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.127304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca372a-e4f1-4e10-9741-fcd59d75cb73 None admin] PATH_INFO: `/v3/groups` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.127923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca372a-e4f1-4e10-9741-fcd59d75cb73 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.129483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca372a-e4f1-4e10-9741-fcd59d75cb73 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.147576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 158/306] 10.222.0.78 () {60 vars in 1134 bytes} [Tue Aug 30 00:48:03 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:04.148395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca372a-e4f1-4e10-9741-fcd59d75cb73 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.154223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.163516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41856c-93ae-4e22-badf-0f45607cba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.164610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41856c-93ae-4e22-badf-0f45607cba5d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.165101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41856c-93ae-4e22-badf-0f45607cba5d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.165708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41856c-93ae-4e22-badf-0f45607cba5d None admin] PATH_INFO: `/v3/users/4202a006245b49f98d3d843742129a52` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.182252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41856c-93ae-4e22-badf-0f45607cba5d None admin] RBAC: Authorizing `identity:get_user(user_id=4202a006245b49f98d3d843742129a52)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.184092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41856c-93ae-4e22-badf-0f45607cba5d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.186225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 149/307] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/users/4202a006245b49f98d3d843742129a52 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:04.193360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41856c-93ae-4e22-badf-0f45607cba5d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.297142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca372a-e4f1-4e10-9741-fcd59d75cb73 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.325368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.327480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41856c-93ae-4e22-badf-0f45607cba5d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.340108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8990cb-327b-4aaa-ac5f-9cae83aa361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.340931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8990cb-327b-4aaa-ac5f-9cae83aa361a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.341038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8990cb-327b-4aaa-ac5f-9cae83aa361a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.341368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8990cb-327b-4aaa-ac5f-9cae83aa361a None admin] PATH_INFO: `/v3/users/e23fa84869b64ac798c7961d538fba97` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.342524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8990cb-327b-4aaa-ac5f-9cae83aa361a None admin] RBAC: Authorizing `identity:get_user(user_id=e23fa84869b64ac798c7961d538fba97)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.343984 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8990cb-327b-4aaa-ac5f-9cae83aa361a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.346434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 159/308] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/users/e23fa84869b64ac798c7961d538fba97 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:04.347604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842fc50-1114-4f43-937a-a574f27ffe43 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.347889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842fc50-1114-4f43-937a-a574f27ffe43 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.348160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842fc50-1114-4f43-937a-a574f27ffe43 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.349400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 160/309] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:04.354240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842fc50-1114-4f43-937a-a574f27ffe43 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.368871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9f3fc-7a05-4e81-ac0d-a2009cba9525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.369512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9f3fc-7a05-4e81-ac0d-a2009cba9525 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.369855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9f3fc-7a05-4e81-ac0d-a2009cba9525 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.370134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9f3fc-7a05-4e81-ac0d-a2009cba9525 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.370906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9f3fc-7a05-4e81-ac0d-a2009cba9525 None admin] RBAC: Adding query filter params (role.id=69d0a4169025430084668f95838669b1, user.id=4202a006245b49f98d3d843742129a52, scope.system=all) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:04.371220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9f3fc-7a05-4e81-ac0d-a2009cba9525 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.372639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9f3fc-7a05-4e81-ac0d-a2009cba9525 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.378987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 150/310] 10.222.0.78 () {56 vars in 1308 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/role_assignments?user.id=4202a006245b49f98d3d843742129a52&scope.system=all&role.id=69d0a4169025430084668f95838669b1 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:04.380161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22949cb-eef1-40f4-bf2e-9937df54dfc8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.380497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22949cb-eef1-40f4-bf2e-9937df54dfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.380773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22949cb-eef1-40f4-bf2e-9937df54dfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.401196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a22949cb-eef1-40f4-bf2e-9937df54dfc8 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:04.402484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a22949cb-eef1-40f4-bf2e-9937df54dfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:04.459519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.484685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 151/311] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:04.509708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842fc50-1114-4f43-937a-a574f27ffe43 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.546618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc55f-ccb5-43c8-9b4c-de9a8904ccc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.548221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc55f-ccb5-43c8-9b4c-de9a8904ccc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.548552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc55f-ccb5-43c8-9b4c-de9a8904ccc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.548830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc55f-ccb5-43c8-9b4c-de9a8904ccc0 None admin] PATH_INFO: `/v3/system/users/e23fa84869b64ac798c7961d538fba97/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.550399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdc55f-ccb5-43c8-9b4c-de9a8904ccc0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e23fa84869b64ac798c7961d538fba97, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.551782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdc55f-ccb5-43c8-9b4c-de9a8904ccc0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.562772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 161/312] 10.222.0.78 () {58 vars in 1334 bytes} [Tue Aug 30 00:48:04 2022] PUT /identity/v3/system/users/e23fa84869b64ac798c7961d538fba97/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:04.762446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22949cb-eef1-40f4-bf2e-9937df54dfc8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.867087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22949cb-eef1-40f4-bf2e-9937df54dfc8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.907271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317251-465e-4802-9885-0f0c5471a524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.908315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317251-465e-4802-9885-0f0c5471a524 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.908813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317251-465e-4802-9885-0f0c5471a524 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.909364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317251-465e-4802-9885-0f0c5471a524 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.921334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13317251-465e-4802-9885-0f0c5471a524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-13317251-465e-4802-9885-0f0c5471a524 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:04.926493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:04.927649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:04.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 152/313] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:04.933371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc55f-ccb5-43c8-9b4c-de9a8904ccc0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.991431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:05.047201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc55f-ccb5-43c8-9b4c-de9a8904ccc0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:05.082884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ed5f0-25f2-436f-bed1-e21a47786660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:05.083492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ed5f0-25f2-436f-bed1-e21a47786660 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:05.083825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ed5f0-25f2-436f-bed1-e21a47786660 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:05.084105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ed5f0-25f2-436f-bed1-e21a47786660 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:05.084765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ed5f0-25f2-436f-bed1-e21a47786660 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:05.085064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ed5f0-25f2-436f-bed1-e21a47786660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:05.086454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ed5f0-25f2-436f-bed1-e21a47786660 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:05.096806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 162/314] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:05.102322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317251-465e-4802-9885-0f0c5471a524 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:05.152282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:05.214501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317251-465e-4802-9885-0f0c5471a524 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:05.261103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc70cd-3b22-4769-98de-04acff511672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:05.261103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc70cd-3b22-4769-98de-04acff511672 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:05.261103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc70cd-3b22-4769-98de-04acff511672 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:05.261103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc70cd-3b22-4769-98de-04acff511672 None admin] PATH_INFO: `/v3/users/ace213c5f27745f2b31f647551104cc0` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:05.261753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc70cd-3b22-4769-98de-04acff511672 None admin] RBAC: Authorizing `identity:get_user(user_id=ace213c5f27745f2b31f647551104cc0)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:05.261895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc70cd-3b22-4769-98de-04acff511672 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:05.263703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 153/315] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:05 2022] GET /identity/v3/users/ace213c5f27745f2b31f647551104cc0 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:05.269847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ed5f0-25f2-436f-bed1-e21a47786660 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:05.376420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ed5f0-25f2-436f-bed1-e21a47786660 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:05.393628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:05.417211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7c6c5-2785-4b36-a439-39790ba205d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:05.417827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7c6c5-2785-4b36-a439-39790ba205d8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:05.418107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7c6c5-2785-4b36-a439-39790ba205d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:05.418381 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7c6c5-2785-4b36-a439-39790ba205d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:05.488926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7c6c5-2785-4b36-a439-39790ba205d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:05.490447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7c6c5-2785-4b36-a439-39790ba205d8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:05.595258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 163/316] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:05.600911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc70cd-3b22-4769-98de-04acff511672 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:05.706699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc70cd-3b22-4769-98de-04acff511672 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:05.717594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:05.742164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397e90c-37f6-457e-8f02-8af7cb1bf0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:05.742819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397e90c-37f6-457e-8f02-8af7cb1bf0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:05.743100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397e90c-37f6-457e-8f02-8af7cb1bf0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:05.743375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397e90c-37f6-457e-8f02-8af7cb1bf0b2 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:05.744619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397e90c-37f6-457e-8f02-8af7cb1bf0b2 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:05.746124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397e90c-37f6-457e-8f02-8af7cb1bf0b2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:05.747880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 154/317] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:05 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:05.754976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7c6c5-2785-4b36-a439-39790ba205d8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:05.777453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:05.859518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7c6c5-2785-4b36-a439-39790ba205d8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:05.895287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3450e-7cc4-4c12-a314-04369338a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:05.895895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3450e-7cc4-4c12-a314-04369338a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:05.896175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3450e-7cc4-4c12-a314-04369338a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:05.896446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3450e-7cc4-4c12-a314-04369338a4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:05.897198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3450e-7cc4-4c12-a314-04369338a4f4 None admin] RBAC: Adding query filter params (role.id=69d0a4169025430084668f95838669b1, scope.project.id=b5a0684693f546a9b65d6dd77f36a932, user.id=ace213c5f27745f2b31f647551104cc0) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:05.897506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3450e-7cc4-4c12-a314-04369338a4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:05.898955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3450e-7cc4-4c12-a314-04369338a4f4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:05.905156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 164/318] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:05 2022] GET /identity/v3/role_assignments?user.id=ace213c5f27745f2b31f647551104cc0&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=69d0a4169025430084668f95838669b1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:06.150141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d092da-6d02-494f-bc6e-1e793fe8502c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.150515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d092da-6d02-494f-bc6e-1e793fe8502c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.150693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d092da-6d02-494f-bc6e-1e793fe8502c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.152157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 155/319] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:06.157574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ea73c-bc77-4eba-bd44-d2a5a31dc7b9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.158239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ea73c-bc77-4eba-bd44-d2a5a31dc7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.158239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ea73c-bc77-4eba-bd44-d2a5a31dc7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.177162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a15ea73c-bc77-4eba-bd44-d2a5a31dc7b9 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:06.178355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a15ea73c-bc77-4eba-bd44-d2a5a31dc7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:06.251365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 165/320] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:06.427437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0752d-78a8-4211-9826-96622eb5daf6 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.427732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0752d-78a8-4211-9826-96622eb5daf6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.427897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0752d-78a8-4211-9826-96622eb5daf6 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.429428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 156/321] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:06.434451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669cd7c-03f3-43cc-962e-923e1d82811c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.434738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669cd7c-03f3-43cc-962e-923e1d82811c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.435009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669cd7c-03f3-43cc-962e-923e1d82811c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.454490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5669cd7c-03f3-43cc-962e-923e1d82811c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:06.456003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5669cd7c-03f3-43cc-962e-923e1d82811c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:06.464074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:06.523513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0752d-78a8-4211-9826-96622eb5daf6 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:06.526117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 166/322] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:06.593885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24e045-0361-44f2-a29e-2e0235654f3a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.594172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24e045-0361-44f2-a29e-2e0235654f3a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.595088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24e045-0361-44f2-a29e-2e0235654f3a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.596701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 167/323] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:06.601577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3a0c-0ee5-43c9-8d39-15c0d228405e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.601857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3a0c-0ee5-43c9-8d39-15c0d228405e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.602133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3a0c-0ee5-43c9-8d39-15c0d228405e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.619900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b24b3a0c-0ee5-43c9-8d39-15c0d228405e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:06.621090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b24b3a0c-0ee5-43c9-8d39-15c0d228405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:06.636184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0752d-78a8-4211-9826-96622eb5daf6 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:06.675519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839de3b-bbdb-4351-9108-8eec62b3cdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:06.676519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839de3b-bbdb-4351-9108-8eec62b3cdfc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.676859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839de3b-bbdb-4351-9108-8eec62b3cdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.677242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839de3b-bbdb-4351-9108-8eec62b3cdfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.678831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839de3b-bbdb-4351-9108-8eec62b3cdfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:06.681049 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839de3b-bbdb-4351-9108-8eec62b3cdfc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:06.683821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 157/324] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:06.691475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839de3b-bbdb-4351-9108-8eec62b3cdfc None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:06.706755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 168/325] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:06.742591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:06.796518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b3a0c-0ee5-43c9-8d39-15c0d228405e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:06.797322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839de3b-bbdb-4351-9108-8eec62b3cdfc None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:06.835672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6147a8-43c5-4442-954e-b2c64aec0b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:06.836274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6147a8-43c5-4442-954e-b2c64aec0b45 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.836552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6147a8-43c5-4442-954e-b2c64aec0b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.836824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6147a8-43c5-4442-954e-b2c64aec0b45 None admin] PATH_INFO: `/v3/groups` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.837491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6147a8-43c5-4442-954e-b2c64aec0b45 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:06.838931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6147a8-43c5-4442-954e-b2c64aec0b45 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:06.865111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 158/326] 10.222.0.78 () {60 vars in 1134 bytes} [Tue Aug 30 00:48:06 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:48:06.894078 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:06.928718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b3a0c-0ee5-43c9-8d39-15c0d228405e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:06.966423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23856b-38b4-4032-bedc-1d6ac803f5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:06.966962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23856b-38b4-4032-bedc-1d6ac803f5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.967232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23856b-38b4-4032-bedc-1d6ac803f5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.967497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23856b-38b4-4032-bedc-1d6ac803f5e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.978670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23856b-38b4-4032-bedc-1d6ac803f5e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-5f23856b-38b4-4032-bedc-1d6ac803f5e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:06.983586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:06.985412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:06.986967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 169/327] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:06.990894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6147a8-43c5-4442-954e-b2c64aec0b45 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.003057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23856b-38b4-4032-bedc-1d6ac803f5e7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.123797 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6147a8-43c5-4442-954e-b2c64aec0b45 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.129134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23856b-38b4-4032-bedc-1d6ac803f5e7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.139336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.166954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.171959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b7c42-a2c3-4e52-a022-1b1c1527b9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.172819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68881c7-684a-4338-a562-e2a8b7e240e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.173275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b7c42-a2c3-4e52-a022-1b1c1527b9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.173508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68881c7-684a-4338-a562-e2a8b7e240e3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.173605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b7c42-a2c3-4e52-a022-1b1c1527b9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.173793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68881c7-684a-4338-a562-e2a8b7e240e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.173886 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b7c42-a2c3-4e52-a022-1b1c1527b9e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.174069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68881c7-684a-4338-a562-e2a8b7e240e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.174750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68881c7-684a-4338-a562-e2a8b7e240e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:07.175056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68881c7-684a-4338-a562-e2a8b7e240e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:07.176629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68881c7-684a-4338-a562-e2a8b7e240e3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:07.185335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b7c42-a2c3-4e52-a022-1b1c1527b9e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:07.187159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 159/328] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4e3b7c42-a2c3-4e52-a022-1b1c1527b9e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:07.191032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:07.192156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:07.193190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 170/329] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:07.193961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68881c7-684a-4338-a562-e2a8b7e240e3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.202054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b7c42-a2c3-4e52-a022-1b1c1527b9e0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.324746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68881c7-684a-4338-a562-e2a8b7e240e3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.342089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b7c42-a2c3-4e52-a022-1b1c1527b9e0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.372785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0190a40-d0a6-4078-8251-a11a612cbc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.373576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0190a40-d0a6-4078-8251-a11a612cbc7a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.373858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0190a40-d0a6-4078-8251-a11a612cbc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.374137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0190a40-d0a6-4078-8251-a11a612cbc7a None admin] PATH_INFO: `/v3/users/4202a006245b49f98d3d843742129a52` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.376418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0190a40-d0a6-4078-8251-a11a612cbc7a None admin] RBAC: Authorizing `identity:get_user(user_id=4202a006245b49f98d3d843742129a52)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:07.377848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0190a40-d0a6-4078-8251-a11a612cbc7a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:07.381293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 160/330] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/users/4202a006245b49f98d3d843742129a52 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:07.385487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0190a40-d0a6-4078-8251-a11a612cbc7a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.394540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad9c47-19a9-4bfb-9125-31abe84b2bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.395149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad9c47-19a9-4bfb-9125-31abe84b2bca None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.395417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad9c47-19a9-4bfb-9125-31abe84b2bca None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.395679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad9c47-19a9-4bfb-9125-31abe84b2bca None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.396937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad9c47-19a9-4bfb-9125-31abe84b2bca None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:07.397309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad9c47-19a9-4bfb-9125-31abe84b2bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:07.399158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad9c47-19a9-4bfb-9125-31abe84b2bca None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:07.413169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 171/331] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:07.417739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad9c47-19a9-4bfb-9125-31abe84b2bca None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.434141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.502074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.536350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0190a40-d0a6-4078-8251-a11a612cbc7a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.576565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad9c47-19a9-4bfb-9125-31abe84b2bca None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.586088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcace48-194f-4eff-a429-7a14cd7c22a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.587111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcace48-194f-4eff-a429-7a14cd7c22a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.587609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcace48-194f-4eff-a429-7a14cd7c22a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.588080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcace48-194f-4eff-a429-7a14cd7c22a1 None admin] PATH_INFO: `/v3/system/users/4202a006245b49f98d3d843742129a52/roles/69d0a4169025430084668f95838669b1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.590467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcace48-194f-4eff-a429-7a14cd7c22a1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4202a006245b49f98d3d843742129a52, role_id=69d0a4169025430084668f95838669b1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:07.592624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcace48-194f-4eff-a429-7a14cd7c22a1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:07.607967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 161/332] 10.222.0.78 () {58 vars in 1334 bytes} [Tue Aug 30 00:48:07 2022] PUT /identity/v3/system/users/4202a006245b49f98d3d843742129a52/roles/69d0a4169025430084668f95838669b1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:48:07.634892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca0119-a726-44a6-b93a-21dfe5296b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.635550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca0119-a726-44a6-b93a-21dfe5296b4b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.635827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca0119-a726-44a6-b93a-21dfe5296b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.636095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca0119-a726-44a6-b93a-21dfe5296b4b None admin] PATH_INFO: `/v3/users/e23fa84869b64ac798c7961d538fba97` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.638581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca0119-a726-44a6-b93a-21dfe5296b4b None admin] RBAC: Authorizing `identity:get_user(user_id=e23fa84869b64ac798c7961d538fba97)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:07.640134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca0119-a726-44a6-b93a-21dfe5296b4b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:07.642577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 172/333] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/users/e23fa84869b64ac798c7961d538fba97 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:07.648574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcace48-194f-4eff-a429-7a14cd7c22a1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.790266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.799660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcace48-194f-4eff-a429-7a14cd7c22a1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.839907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a4181-b1e2-499d-a20f-8395a907d36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.840879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a4181-b1e2-499d-a20f-8395a907d36a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.841409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a4181-b1e2-499d-a20f-8395a907d36a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.841890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a4181-b1e2-499d-a20f-8395a907d36a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.842825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a4181-b1e2-499d-a20f-8395a907d36a None admin] RBAC: Adding query filter params (role.id=e407aec03c22425591fa0e52a40edbc1, user.id=e23fa84869b64ac798c7961d538fba97, scope.system=all) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:07.843336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a4181-b1e2-499d-a20f-8395a907d36a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:07.845028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a4181-b1e2-499d-a20f-8395a907d36a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:07.858595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 162/334] 10.222.0.78 () {56 vars in 1308 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/role_assignments?user.id=e23fa84869b64ac798c7961d538fba97&scope.system=all&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:09.192904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81c195-fbf7-4f88-8b31-95801b4297f3 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:09.193453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81c195-fbf7-4f88-8b31-95801b4297f3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:09.193453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81c195-fbf7-4f88-8b31-95801b4297f3 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:09.195338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 173/335] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:09.200579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30228472-f571-40c1-8104-5de860b64868 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:09.201405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30228472-f571-40c1-8104-5de860b64868 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:09.201882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30228472-f571-40c1-8104-5de860b64868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:09.224252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-30228472-f571-40c1-8104-5de860b64868 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:09.225796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-30228472-f571-40c1-8104-5de860b64868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:09.316495 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 163/336] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:09.540026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae63849-9e4f-4fee-b07f-649ddb4345fa None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:09.540368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae63849-9e4f-4fee-b07f-649ddb4345fa None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:09.540564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae63849-9e4f-4fee-b07f-649ddb4345fa None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:09.542013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 174/337] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:09.547290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309833b1-8ee6-453d-bebe-53f702194761 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:09.548527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309833b1-8ee6-453d-bebe-53f702194761 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:09.549113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309833b1-8ee6-453d-bebe-53f702194761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:09.578211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-309833b1-8ee6-453d-bebe-53f702194761 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:09.580546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-309833b1-8ee6-453d-bebe-53f702194761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:09.656221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 164/338] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:09.876456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae63849-9e4f-4fee-b07f-649ddb4345fa None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:09.994714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae63849-9e4f-4fee-b07f-649ddb4345fa None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:10.023895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:10.037663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02876c51-e734-4c3b-b4c1-6cd499a2b7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:10.038618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02876c51-e734-4c3b-b4c1-6cd499a2b7aa None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:10.039099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02876c51-e734-4c3b-b4c1-6cd499a2b7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:10.039570 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02876c51-e734-4c3b-b4c1-6cd499a2b7aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:10.052196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02876c51-e734-4c3b-b4c1-6cd499a2b7aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-02876c51-e734-4c3b-b4c1-6cd499a2b7aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:10.057341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:10.058231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:10.059026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:10.065678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 175/339] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:10.071433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309833b1-8ee6-453d-bebe-53f702194761 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:10.091618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:10.202907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309833b1-8ee6-453d-bebe-53f702194761 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:10.238215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255349e2-2235-4abc-a549-f38103560e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:10.238830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255349e2-2235-4abc-a549-f38103560e2f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:10.239106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255349e2-2235-4abc-a549-f38103560e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:10.239371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255349e2-2235-4abc-a549-f38103560e2f None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:10.242204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255349e2-2235-4abc-a549-f38103560e2f None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:10.242204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255349e2-2235-4abc-a549-f38103560e2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:10.242204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255349e2-2235-4abc-a549-f38103560e2f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:10.253223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 165/340] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:10.263202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02876c51-e734-4c3b-b4c1-6cd499a2b7aa None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:10.321808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255349e2-2235-4abc-a549-f38103560e2f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:10.414514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02876c51-e734-4c3b-b4c1-6cd499a2b7aa None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:10.461999 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:10.493812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432e942-6394-464a-bf43-4003ed68f0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:10.494445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432e942-6394-464a-bf43-4003ed68f0ce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:10.494728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432e942-6394-464a-bf43-4003ed68f0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:10.495000 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432e942-6394-464a-bf43-4003ed68f0ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:10.519044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255349e2-2235-4abc-a549-f38103560e2f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:10.573474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:10.578931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbc6be-acf4-4091-8f99-e63d6350edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:10.579692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbc6be-acf4-4091-8f99-e63d6350edad None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:10.580032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbc6be-acf4-4091-8f99-e63d6350edad None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:10.580289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbc6be-acf4-4091-8f99-e63d6350edad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:10.597572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432e942-6394-464a-bf43-4003ed68f0ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:10.599041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432e942-6394-464a-bf43-4003ed68f0ce None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:10.605398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbc6be-acf4-4091-8f99-e63d6350edad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7ffbc6be-acf4-4091-8f99-e63d6350edad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:10.611405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:10.612317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:10.613139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 166/341] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:10.614294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37002c29-a295-4837-9b84-88bba7520022 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:10.614389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37002c29-a295-4837-9b84-88bba7520022 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:10.615537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37002c29-a295-4837-9b84-88bba7520022 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:10.619477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 167/342] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:10.620842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37002c29-a295-4837-9b84-88bba7520022 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:10.776216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37002c29-a295-4837-9b84-88bba7520022 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:10.800061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 176/343] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:10.804370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea94d9-be61-44b0-9642-48a0a79b9c38 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:10.804370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea94d9-be61-44b0-9642-48a0a79b9c38 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:10.804370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea94d9-be61-44b0-9642-48a0a79b9c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:10.823885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f528b02-0db4-4a32-bf8a-6ef2f9bfda7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:10.824455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f0ea94d9-be61-44b0-9642-48a0a79b9c38 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:10.825150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f528b02-0db4-4a32-bf8a-6ef2f9bfda7e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:10.825652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f0ea94d9-be61-44b0-9642-48a0a79b9c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:10.825868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f528b02-0db4-4a32-bf8a-6ef2f9bfda7e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:10.826338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f528b02-0db4-4a32-bf8a-6ef2f9bfda7e None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:10.827248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f528b02-0db4-4a32-bf8a-6ef2f9bfda7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:10.827754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f528b02-0db4-4a32-bf8a-6ef2f9bfda7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:10.830830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f528b02-0db4-4a32-bf8a-6ef2f9bfda7e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:10.841123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 168/344] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:10.842737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e07d2-f94c-4b1e-9151-c6cd54fa3f97 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:10.843234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e07d2-f94c-4b1e-9151-c6cd54fa3f97 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:10.843507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e07d2-f94c-4b1e-9151-c6cd54fa3f97 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:10.848717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 169/345] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:10.849930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e07d2-f94c-4b1e-9151-c6cd54fa3f97 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:10.853284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:10.916696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 177/346] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:10.917993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54848fc8-210d-4dff-91c2-a1481f18d4c7 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:10.918275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54848fc8-210d-4dff-91c2-a1481f18d4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:10.918542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54848fc8-210d-4dff-91c2-a1481f18d4c7 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:10.919880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 178/347] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:10.920496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54848fc8-210d-4dff-91c2-a1481f18d4c7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:10.941795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:10.987684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e07d2-f94c-4b1e-9151-c6cd54fa3f97 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:11.030629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6879dd-79ad-465e-a737-7b5a73a483cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:11.031174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6879dd-79ad-465e-a737-7b5a73a483cd None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.031451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6879dd-79ad-465e-a737-7b5a73a483cd None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.031772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6879dd-79ad-465e-a737-7b5a73a483cd None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.033389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6879dd-79ad-465e-a737-7b5a73a483cd None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:11.034816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6879dd-79ad-465e-a737-7b5a73a483cd None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:11.036611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 170/348] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:10 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:11.040655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b584b-d5dd-4317-aed6-8af672dda0f0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.040655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b584b-d5dd-4317-aed6-8af672dda0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.040655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b584b-d5dd-4317-aed6-8af672dda0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.062009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-bf1b584b-d5dd-4317-aed6-8af672dda0f0 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:11.063825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bf1b584b-d5dd-4317-aed6-8af672dda0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:11.078323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54848fc8-210d-4dff-91c2-a1481f18d4c7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:11.128525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4f7a4-91b9-4980-9d87-6ed1fb19dec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:11.129247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4f7a4-91b9-4980-9d87-6ed1fb19dec2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.129575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4f7a4-91b9-4980-9d87-6ed1fb19dec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.129844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4f7a4-91b9-4980-9d87-6ed1fb19dec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.155546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:11.162570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 171/349] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:11.164109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce1932-e660-4195-bd37-81fb40405bef None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.164593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce1932-e660-4195-bd37-81fb40405bef None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.165155 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce1932-e660-4195-bd37-81fb40405bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.190877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-87ce1932-e660-4195-bd37-81fb40405bef None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:11.192385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-87ce1932-e660-4195-bd37-81fb40405bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:11.221393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4f7a4-91b9-4980-9d87-6ed1fb19dec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:11.223148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4f7a4-91b9-4980-9d87-6ed1fb19dec2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:11.245516 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:11.271839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 172/350] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:11.273882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce1932-e660-4195-bd37-81fb40405bef None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:11.356983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 179/351] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:11.363926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4f7a4-91b9-4980-9d87-6ed1fb19dec2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:11.391835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce1932-e660-4195-bd37-81fb40405bef None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:11.429886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:11.437681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26834c0e-e1fb-41ef-a6f4-a47235500115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:11.438594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834c0e-e1fb-41ef-a6f4-a47235500115 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.439068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834c0e-e1fb-41ef-a6f4-a47235500115 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.439551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26834c0e-e1fb-41ef-a6f4-a47235500115 None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.449945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26834c0e-e1fb-41ef-a6f4-a47235500115 None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:11.452710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26834c0e-e1fb-41ef-a6f4-a47235500115 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:11.455262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 173/352] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:11.457092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26834c0e-e1fb-41ef-a6f4-a47235500115 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:11.514841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4f7a4-91b9-4980-9d87-6ed1fb19dec2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:11.537511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:11.558652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0860935-884e-4f48-89b5-e512dd282d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:11.559279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0860935-884e-4f48-89b5-e512dd282d50 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.559654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0860935-884e-4f48-89b5-e512dd282d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.559925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0860935-884e-4f48-89b5-e512dd282d50 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.561160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0860935-884e-4f48-89b5-e512dd282d50 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:11.562591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0860935-884e-4f48-89b5-e512dd282d50 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:11.564655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 180/353] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:11.565681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0860935-884e-4f48-89b5-e512dd282d50 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:11.670578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26834c0e-e1fb-41ef-a6f4-a47235500115 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:11.688700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0860935-884e-4f48-89b5-e512dd282d50 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:11.707943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009298c-fe0d-4f10-9431-20dfbafe0e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:11.708935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009298c-fe0d-4f10-9431-20dfbafe0e25 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.709307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009298c-fe0d-4f10-9431-20dfbafe0e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.709598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009298c-fe0d-4f10-9431-20dfbafe0e25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.722502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009298c-fe0d-4f10-9431-20dfbafe0e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-8009298c-fe0d-4f10-9431-20dfbafe0e25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:11.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:11.728680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:11.729530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 174/354] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:11.730196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009298c-fe0d-4f10-9431-20dfbafe0e25 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:11.730497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb0491-4403-4df3-8f38-49954939998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:11.731521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb0491-4403-4df3-8f38-49954939998b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.731521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb0491-4403-4df3-8f38-49954939998b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.731521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb0491-4403-4df3-8f38-49954939998b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.732328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb0491-4403-4df3-8f38-49954939998b None admin] RBAC: Adding query filter params (group.id=6cb198ecbe1a4f2480de34cc9bd7e839, role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:11.732645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb0491-4403-4df3-8f38-49954939998b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:11.735576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb0491-4403-4df3-8f38-49954939998b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:11.741719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 181/355] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/role_assignments?group.id=6cb198ecbe1a4f2480de34cc9bd7e839&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:11.742518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb0491-4403-4df3-8f38-49954939998b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:11.803759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:11.815323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:11.885866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb0491-4403-4df3-8f38-49954939998b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:11.894147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009298c-fe0d-4f10-9431-20dfbafe0e25 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:11.929159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4287e6-3b92-44f5-b884-aabef031f93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:11.929720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4287e6-3b92-44f5-b884-aabef031f93a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.929995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4287e6-3b92-44f5-b884-aabef031f93a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.930259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4287e6-3b92-44f5-b884-aabef031f93a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.938059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424270f-3f9e-44ff-b90e-3f6537b4bb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:11.938688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424270f-3f9e-44ff-b90e-3f6537b4bb4c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.938963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424270f-3f9e-44ff-b90e-3f6537b4bb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.939229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424270f-3f9e-44ff-b90e-3f6537b4bb4c None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.940448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424270f-3f9e-44ff-b90e-3f6537b4bb4c None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:11.941903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424270f-3f9e-44ff-b90e-3f6537b4bb4c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:11.943627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4287e6-3b92-44f5-b884-aabef031f93a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:11.943735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 175/356] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:11.948966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424270f-3f9e-44ff-b90e-3f6537b4bb4c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-fb4287e6-3b92-44f5-b884-aabef031f93a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:11.949435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:11.950296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:11.951115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 182/357] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:11.951771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02419ec3-32db-464a-ae25-9e53f43a35d5 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.952482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02419ec3-32db-464a-ae25-9e53f43a35d5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.952482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02419ec3-32db-464a-ae25-9e53f43a35d5 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.959443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 183/358] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:11.960160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02419ec3-32db-464a-ae25-9e53f43a35d5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.095244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02419ec3-32db-464a-ae25-9e53f43a35d5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.099815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.105177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424270f-3f9e-44ff-b90e-3f6537b4bb4c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.132470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.137962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8cdca-25eb-4a4e-a070-c6d4886534ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.139181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8cdca-25eb-4a4e-a070-c6d4886534ac None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.139181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8cdca-25eb-4a4e-a070-c6d4886534ac None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.139181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8cdca-25eb-4a4e-a070-c6d4886534ac None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.139656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8cdca-25eb-4a4e-a070-c6d4886534ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:12.139899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8cdca-25eb-4a4e-a070-c6d4886534ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.142495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8cdca-25eb-4a4e-a070-c6d4886534ac None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.150006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baad48-3e50-47aa-8957-79c47444ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.150644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baad48-3e50-47aa-8957-79c47444ed66 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.150925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baad48-3e50-47aa-8957-79c47444ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.151199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baad48-3e50-47aa-8957-79c47444ed66 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.152010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 184/359] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:12.152682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8cdca-25eb-4a4e-a070-c6d4886534ac None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.164688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baad48-3e50-47aa-8957-79c47444ed66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c2baad48-3e50-47aa-8957-79c47444ed66 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:12.170178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:12.171075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 176/360] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:12.172051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baad48-3e50-47aa-8957-79c47444ed66 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.290211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8cdca-25eb-4a4e-a070-c6d4886534ac None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.314535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baad48-3e50-47aa-8957-79c47444ed66 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.338745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d3efc-5c3b-40b2-95c5-300016c1efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.339350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d3efc-5c3b-40b2-95c5-300016c1efb1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.339637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d3efc-5c3b-40b2-95c5-300016c1efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.339901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d3efc-5c3b-40b2-95c5-300016c1efb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.340630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d3efc-5c3b-40b2-95c5-300016c1efb1 None admin] RBAC: Adding query filter params (group.id=6cb198ecbe1a4f2480de34cc9bd7e839, role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:12.340935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d3efc-5c3b-40b2-95c5-300016c1efb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.342442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d3efc-5c3b-40b2-95c5-300016c1efb1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.350501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 185/361] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/role_assignments?group.id=6cb198ecbe1a4f2480de34cc9bd7e839&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:12.351793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55420f-1725-470b-88b1-d321521d0d08 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.351793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55420f-1725-470b-88b1-d321521d0d08 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.352066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55420f-1725-470b-88b1-d321521d0d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.362912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3603-9d17-439e-a019-93eb5fd70a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.363533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3603-9d17-439e-a019-93eb5fd70a69 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.363809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3603-9d17-439e-a019-93eb5fd70a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.364073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3603-9d17-439e-a019-93eb5fd70a69 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.364748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3603-9d17-439e-a019-93eb5fd70a69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:12.365047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3603-9d17-439e-a019-93eb5fd70a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.366652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3603-9d17-439e-a019-93eb5fd70a69 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.375719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0b55420f-1725-470b-88b1-d321521d0d08 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:12.376602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 177/362] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:12.377065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3603-9d17-439e-a019-93eb5fd70a69 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.378392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0b55420f-1725-470b-88b1-d321521d0d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:12.413057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.432209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.482223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 186/363] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:12.487221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55420f-1725-470b-88b1-d321521d0d08 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.553401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3603-9d17-439e-a019-93eb5fd70a69 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.614338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f426167-a269-4f4d-b1e2-f0c333185fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.615031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f426167-a269-4f4d-b1e2-f0c333185fec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.615310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f426167-a269-4f4d-b1e2-f0c333185fec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.615575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f426167-a269-4f4d-b1e2-f0c333185fec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.632439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55420f-1725-470b-88b1-d321521d0d08 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.672357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd1eca-5d89-45de-9da2-676107ab61be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.673286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd1eca-5d89-45de-9da2-676107ab61be None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.673712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd1eca-5d89-45de-9da2-676107ab61be None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.674215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd1eca-5d89-45de-9da2-676107ab61be None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.675085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd1eca-5d89-45de-9da2-676107ab61be None admin] RBAC: Adding query filter params (name=reader) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:12.675527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd1eca-5d89-45de-9da2-676107ab61be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.677058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd1eca-5d89-45de-9da2-676107ab61be None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.686558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 187/364] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:12.687696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd1eca-5d89-45de-9da2-676107ab61be None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.721157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.744867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f426167-a269-4f4d-b1e2-f0c333185fec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.749990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f426167-a269-4f4d-b1e2-f0c333185fec None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.832695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.858485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd1eca-5d89-45de-9da2-676107ab61be None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.911804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c41e2-a128-45fe-afca-01b290d8254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.912681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c41e2-a128-45fe-afca-01b290d8254f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.913126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c41e2-a128-45fe-afca-01b290d8254f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.913568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c41e2-a128-45fe-afca-01b290d8254f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.921892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 178/365] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:12.922995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f426167-a269-4f4d-b1e2-f0c333185fec None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.994913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c41e2-a128-45fe-afca-01b290d8254f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.996677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c41e2-a128-45fe-afca-01b290d8254f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:13.029150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:13.073840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f426167-a269-4f4d-b1e2-f0c333185fec None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:13.115559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a310813-6861-404f-bddf-b5338ac9fd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:13.116320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a310813-6861-404f-bddf-b5338ac9fd60 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:13.116513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a310813-6861-404f-bddf-b5338ac9fd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:13.116784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a310813-6861-404f-bddf-b5338ac9fd60 None admin] PATH_INFO: `/v3/users/4202a006245b49f98d3d843742129a52` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:13.117937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a310813-6861-404f-bddf-b5338ac9fd60 None admin] RBAC: Authorizing `identity:get_user(user_id=4202a006245b49f98d3d843742129a52)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:13.119459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a310813-6861-404f-bddf-b5338ac9fd60 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:13.121445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 179/366] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/users/4202a006245b49f98d3d843742129a52 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:13.122205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a310813-6861-404f-bddf-b5338ac9fd60 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:13.131722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 188/367] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:13.132803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c41e2-a128-45fe-afca-01b290d8254f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:13.149038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:13.269728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a310813-6861-404f-bddf-b5338ac9fd60 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:13.269816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c41e2-a128-45fe-afca-01b290d8254f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:13.306045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:13.318279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c0a00-049d-4736-99f9-92747b71640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:13.318987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c0a00-049d-4736-99f9-92747b71640b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:13.319197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c0a00-049d-4736-99f9-92747b71640b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:13.319723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c0a00-049d-4736-99f9-92747b71640b None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:13.320842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c0a00-049d-4736-99f9-92747b71640b None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:13.322303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c0a00-049d-4736-99f9-92747b71640b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:13.323607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b9044-4c2d-44ff-ac05-662eb57df3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:13.324254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 180/368] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:13 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:13.324678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b9044-4c2d-44ff-ac05-662eb57df3bb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:13.325023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c0a00-049d-4736-99f9-92747b71640b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:13.325390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b9044-4c2d-44ff-ac05-662eb57df3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:13.326045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b9044-4c2d-44ff-ac05-662eb57df3bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:13.341253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b9044-4c2d-44ff-ac05-662eb57df3bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3d0b9044-4c2d-44ff-ac05-662eb57df3bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:13.348647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:13.349738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:13.350557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:13.353587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 189/369] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:13.354767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b9044-4c2d-44ff-ac05-662eb57df3bb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:13.421012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:13.444330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c0a00-049d-4736-99f9-92747b71640b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:13.470900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b9044-4c2d-44ff-ac05-662eb57df3bb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:13.482790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25a4b8-a93a-4dd9-8bb5-1589b63d579a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:13.483395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25a4b8-a93a-4dd9-8bb5-1589b63d579a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:13.483671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25a4b8-a93a-4dd9-8bb5-1589b63d579a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:13.483938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25a4b8-a93a-4dd9-8bb5-1589b63d579a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:13.484680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25a4b8-a93a-4dd9-8bb5-1589b63d579a None admin] RBAC: Adding query filter params (role.id=69d0a4169025430084668f95838669b1, user.id=4202a006245b49f98d3d843742129a52, scope.system=all) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:13.485781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25a4b8-a93a-4dd9-8bb5-1589b63d579a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:13.487561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25a4b8-a93a-4dd9-8bb5-1589b63d579a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:13.501743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 181/370] 10.222.0.78 () {56 vars in 1308 bytes} [Tue Aug 30 00:48:13 2022] GET /identity/v3/role_assignments?user.id=4202a006245b49f98d3d843742129a52&scope.system=all&role.id=69d0a4169025430084668f95838669b1 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:13.502591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25a4b8-a93a-4dd9-8bb5-1589b63d579a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:13.511505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7051e78-e5ab-4502-a78f-6f171c516871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:13.512480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051e78-e5ab-4502-a78f-6f171c516871 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:13.512976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051e78-e5ab-4502-a78f-6f171c516871 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:13.513471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051e78-e5ab-4502-a78f-6f171c516871 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:13.514795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7051e78-e5ab-4502-a78f-6f171c516871 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:13.517045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7051e78-e5ab-4502-a78f-6f171c516871 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:13.519293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 190/371] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:13 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:13.520334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7051e78-e5ab-4502-a78f-6f171c516871 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:13.594242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:13.676887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7051e78-e5ab-4502-a78f-6f171c516871 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:13.725195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25a4b8-a93a-4dd9-8bb5-1589b63d579a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:13.758648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc89cb6-b06b-4698-a493-faa9a0d8ad5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:13.759369 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc89cb6-b06b-4698-a493-faa9a0d8ad5c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:13.759650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc89cb6-b06b-4698-a493-faa9a0d8ad5c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:13.759919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc89cb6-b06b-4698-a493-faa9a0d8ad5c None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:13.760581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc89cb6-b06b-4698-a493-faa9a0d8ad5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:13.760898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc89cb6-b06b-4698-a493-faa9a0d8ad5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:13.763077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc89cb6-b06b-4698-a493-faa9a0d8ad5c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:13.772919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbbd511-aa73-45ab-82cb-2d126218e0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:13.773886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbd511-aa73-45ab-82cb-2d126218e0db None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:13.774345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbd511-aa73-45ab-82cb-2d126218e0db None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:13.774953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbd511-aa73-45ab-82cb-2d126218e0db None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:13.776432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbbd511-aa73-45ab-82cb-2d126218e0db None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:13.778078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbbd511-aa73-45ab-82cb-2d126218e0db None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:13.778186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 191/372] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:13.779034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc89cb6-b06b-4698-a493-faa9a0d8ad5c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:13.780286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 182/373] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:13 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:13.785046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbbd511-aa73-45ab-82cb-2d126218e0db None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:13.798464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:13.914042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc89cb6-b06b-4698-a493-faa9a0d8ad5c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:13.926975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbbd511-aa73-45ab-82cb-2d126218e0db None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:13.931412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:13.960571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35233-d830-4082-8b04-89683555d066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:13.961235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35233-d830-4082-8b04-89683555d066 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:13.961515 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35233-d830-4082-8b04-89683555d066 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:13.961785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35233-d830-4082-8b04-89683555d066 None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:13.962898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf35233-d830-4082-8b04-89683555d066 None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:13.964321 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf35233-d830-4082-8b04-89683555d066 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:13.966222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 192/374] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:13 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:13.966999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35233-d830-4082-8b04-89683555d066 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:13.975906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7107332-7990-46f1-b7dc-7a10f7506675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:13.976621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107332-7990-46f1-b7dc-7a10f7506675 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:13.976906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107332-7990-46f1-b7dc-7a10f7506675 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:13.977233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107332-7990-46f1-b7dc-7a10f7506675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.071402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7107332-7990-46f1-b7dc-7a10f7506675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:14.073374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7107332-7990-46f1-b7dc-7a10f7506675 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:14.084587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:14.090282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35233-d830-4082-8b04-89683555d066 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:14.145558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40052844-f22e-4178-ae6f-09e765765f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:14.146178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052844-f22e-4178-ae6f-09e765765f9b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.146448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052844-f22e-4178-ae6f-09e765765f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.146713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052844-f22e-4178-ae6f-09e765765f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.147446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40052844-f22e-4178-ae6f-09e765765f9b None admin] RBAC: Adding query filter params (group.id=6cb198ecbe1a4f2480de34cc9bd7e839, role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=b5a0684693f546a9b65d6dd77f36a932) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:14.147749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40052844-f22e-4178-ae6f-09e765765f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:14.149193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40052844-f22e-4178-ae6f-09e765765f9b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:14.155816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 193/375] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:13 2022] GET /identity/v3/role_assignments?group.id=6cb198ecbe1a4f2480de34cc9bd7e839&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:14.157576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb965281-6d31-4673-9247-1c9255af96d5 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.157824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb965281-6d31-4673-9247-1c9255af96d5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.158089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb965281-6d31-4673-9247-1c9255af96d5 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.159636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 194/376] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:14.160454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb965281-6d31-4673-9247-1c9255af96d5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:14.227234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 183/377] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:14.228887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bec0f2-cf7c-441a-a882-4ca39bef901e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.229404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bec0f2-cf7c-441a-a882-4ca39bef901e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.229903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bec0f2-cf7c-441a-a882-4ca39bef901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.241970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:14.258889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f0bec0f2-cf7c-441a-a882-4ca39bef901e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:14.260278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f0bec0f2-cf7c-441a-a882-4ca39bef901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:14.332805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb965281-6d31-4673-9247-1c9255af96d5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:14.344324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 184/378] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:14.345495 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bec0f2-cf7c-441a-a882-4ca39bef901e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:14.382892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b24e7-7923-4a27-80b3-ea85509d9bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:14.383499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b24e7-7923-4a27-80b3-ea85509d9bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.383774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b24e7-7923-4a27-80b3-ea85509d9bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.384040 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b24e7-7923-4a27-80b3-ea85509d9bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.384766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b24e7-7923-4a27-80b3-ea85509d9bc2 None admin] RBAC: Adding query filter params (group.id=6cb198ecbe1a4f2480de34cc9bd7e839, role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=b5a0684693f546a9b65d6dd77f36a932) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:14.385067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b24e7-7923-4a27-80b3-ea85509d9bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:14.386531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b24e7-7923-4a27-80b3-ea85509d9bc2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:14.389289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:14.393988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 195/379] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:14 2022] GET /identity/v3/role_assignments?group.id=6cb198ecbe1a4f2480de34cc9bd7e839&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:14.469636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bec0f2-cf7c-441a-a882-4ca39bef901e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:14.510850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d40df5-ecc7-4b2a-9e8b-3d824218b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:14.511414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40df5-ecc7-4b2a-9e8b-3d824218b1fe None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.511691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40df5-ecc7-4b2a-9e8b-3d824218b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.511959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40df5-ecc7-4b2a-9e8b-3d824218b1fe None admin] PATH_INFO: `/v3/projects/761e332e13aa43bb851c08b247de0d02` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.513143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d40df5-ecc7-4b2a-9e8b-3d824218b1fe None admin] RBAC: Authorizing `identity:get_project(project_id=761e332e13aa43bb851c08b247de0d02)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:14.514608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d40df5-ecc7-4b2a-9e8b-3d824218b1fe None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:14.517376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 185/380] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:14 2022] GET /identity/v3/projects/761e332e13aa43bb851c08b247de0d02 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:14.522656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b24e7-7923-4a27-80b3-ea85509d9bc2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:14.643131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b24e7-7923-4a27-80b3-ea85509d9bc2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:14.652356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:14.681131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee155d-efa8-4fc6-b4a2-71fbea93528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:14.681532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee155d-efa8-4fc6-b4a2-71fbea93528a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.681808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee155d-efa8-4fc6-b4a2-71fbea93528a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.682075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee155d-efa8-4fc6-b4a2-71fbea93528a None admin] PATH_INFO: `/v3/groups/b7dc9b6aaf524b42b6a12f628ed42be6` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.682587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d749f94-d549-45e2-9616-2ef059d9b562 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.682885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d749f94-d549-45e2-9616-2ef059d9b562 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.683146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d749f94-d549-45e2-9616-2ef059d9b562 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.684609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 186/381] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:14.687320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d749f94-d549-45e2-9616-2ef059d9b562 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:14.689865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee155d-efa8-4fc6-b4a2-71fbea93528a None admin] RBAC: Authorizing `identity:get_group(group_id=b7dc9b6aaf524b42b6a12f628ed42be6)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:14.691474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee155d-efa8-4fc6-b4a2-71fbea93528a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:14.697416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 196/382] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:14 2022] GET /identity/v3/groups/b7dc9b6aaf524b42b6a12f628ed42be6 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:14.698878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270792-7a87-45c5-88f4-b4db7ce50769 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.699324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270792-7a87-45c5-88f4-b4db7ce50769 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.699735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270792-7a87-45c5-88f4-b4db7ce50769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.721997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-58270792-7a87-45c5-88f4-b4db7ce50769 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:14.723415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-58270792-7a87-45c5-88f4-b4db7ce50769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:14.810355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 197/383] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:14.813141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270792-7a87-45c5-88f4-b4db7ce50769 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:14.835314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d749f94-d549-45e2-9616-2ef059d9b562 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:14.853440 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:14.882248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f501e-0512-41c5-95eb-e299b7b6463a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:14.883296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f501e-0512-41c5-95eb-e299b7b6463a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.883810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f501e-0512-41c5-95eb-e299b7b6463a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.884284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f501e-0512-41c5-95eb-e299b7b6463a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.897635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f501e-0512-41c5-95eb-e299b7b6463a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9d5f501e-0512-41c5-95eb-e299b7b6463a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:14.902935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:14.903880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:14.904653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:14.907719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 187/384] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:14.916236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f501e-0512-41c5-95eb-e299b7b6463a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:14.967302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270792-7a87-45c5-88f4-b4db7ce50769 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:14.968982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:15.038557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898f3d6-9189-421e-84f8-02762bdf6002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.039170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898f3d6-9189-421e-84f8-02762bdf6002 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.039450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898f3d6-9189-421e-84f8-02762bdf6002 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.039717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898f3d6-9189-421e-84f8-02762bdf6002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.040447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898f3d6-9189-421e-84f8-02762bdf6002 None admin] RBAC: Adding query filter params (group.id=b7dc9b6aaf524b42b6a12f628ed42be6, role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=761e332e13aa43bb851c08b247de0d02) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:15.040752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898f3d6-9189-421e-84f8-02762bdf6002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:15.042946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898f3d6-9189-421e-84f8-02762bdf6002 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:15.052409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 198/385] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:14 2022] GET /identity/v3/role_assignments?group.id=b7dc9b6aaf524b42b6a12f628ed42be6&scope.project.id=761e332e13aa43bb851c08b247de0d02&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:15.108301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f501e-0512-41c5-95eb-e299b7b6463a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:15.153324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a26dc-ac49-46bf-9cc1-2d4956dcbd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.153996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a26dc-ac49-46bf-9cc1-2d4956dcbd8a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.154273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a26dc-ac49-46bf-9cc1-2d4956dcbd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.154543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a26dc-ac49-46bf-9cc1-2d4956dcbd8a None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.155218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a26dc-ac49-46bf-9cc1-2d4956dcbd8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:15.155516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a26dc-ac49-46bf-9cc1-2d4956dcbd8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:15.157688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a26dc-ac49-46bf-9cc1-2d4956dcbd8a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:15.170687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 188/386] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:15.177811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898f3d6-9189-421e-84f8-02762bdf6002 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:15.212746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a26dc-ac49-46bf-9cc1-2d4956dcbd8a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:15.256259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:15.339444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a26dc-ac49-46bf-9cc1-2d4956dcbd8a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:15.374327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864f6a2-5761-4bda-a184-076e3e9a8978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.374944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864f6a2-5761-4bda-a184-076e3e9a8978 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.375232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864f6a2-5761-4bda-a184-076e3e9a8978 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.376169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864f6a2-5761-4bda-a184-076e3e9a8978 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.387140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864f6a2-5761-4bda-a184-076e3e9a8978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c864f6a2-5761-4bda-a184-076e3e9a8978 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:15.393328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:15.394181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:15.394971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 189/387] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:15.406051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864f6a2-5761-4bda-a184-076e3e9a8978 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:15.420853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898f3d6-9189-421e-84f8-02762bdf6002 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:15.504961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac5427-46ad-436b-bb7f-c144e797dd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.506032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac5427-46ad-436b-bb7f-c144e797dd21 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.506515 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac5427-46ad-436b-bb7f-c144e797dd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.506988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac5427-46ad-436b-bb7f-c144e797dd21 None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.514273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac5427-46ad-436b-bb7f-c144e797dd21 None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:15.517590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac5427-46ad-436b-bb7f-c144e797dd21 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:15.520967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 199/388] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:15 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:15.526632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac5427-46ad-436b-bb7f-c144e797dd21 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:15.528155 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:15.544544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864f6a2-5761-4bda-a184-076e3e9a8978 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:15.582478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580e9f6-089f-4fc2-ba92-fbbb53d09f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.583102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580e9f6-089f-4fc2-ba92-fbbb53d09f2e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.583390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580e9f6-089f-4fc2-ba92-fbbb53d09f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.583674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580e9f6-089f-4fc2-ba92-fbbb53d09f2e None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.584347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580e9f6-089f-4fc2-ba92-fbbb53d09f2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:15.584655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580e9f6-089f-4fc2-ba92-fbbb53d09f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:15.586066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580e9f6-089f-4fc2-ba92-fbbb53d09f2e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:15.595645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 190/389] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:15.600820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580e9f6-089f-4fc2-ba92-fbbb53d09f2e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:15.600936 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:15.714607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac5427-46ad-436b-bb7f-c144e797dd21 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:15.756162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b2c9-652a-428b-acbc-0d38c1c4678c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.757690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b2c9-652a-428b-acbc-0d38c1c4678c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.758168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b2c9-652a-428b-acbc-0d38c1c4678c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.758674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b2c9-652a-428b-acbc-0d38c1c4678c None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.760007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b2c9-652a-428b-acbc-0d38c1c4678c None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:15.762205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b2c9-652a-428b-acbc-0d38c1c4678c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:15.764328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 200/390] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:15 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:15.771440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b2c9-652a-428b-acbc-0d38c1c4678c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:15.799254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580e9f6-089f-4fc2-ba92-fbbb53d09f2e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:15.889054 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d603eb3-f9bf-4465-833a-3ef2dc6181fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.889715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d603eb3-f9bf-4465-833a-3ef2dc6181fc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.889997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d603eb3-f9bf-4465-833a-3ef2dc6181fc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.890272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d603eb3-f9bf-4465-833a-3ef2dc6181fc None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.891499 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d603eb3-f9bf-4465-833a-3ef2dc6181fc None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:15.893046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d603eb3-f9bf-4465-833a-3ef2dc6181fc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:15.901176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 191/391] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:15 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:15.904242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d603eb3-f9bf-4465-833a-3ef2dc6181fc None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:15.911872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b2c9-652a-428b-acbc-0d38c1c4678c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:15.913760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:15.959080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:15.986434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00791c6b-f969-40fc-9a3a-32f94d20e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.987459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00791c6b-f969-40fc-9a3a-32f94d20e6b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.987946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00791c6b-f969-40fc-9a3a-32f94d20e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.988436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00791c6b-f969-40fc-9a3a-32f94d20e6b1 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/groups/6cb198ecbe1a4f2480de34cc9bd7e839/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.993431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00791c6b-f969-40fc-9a3a-32f94d20e6b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=09da17aa18cc4293b57f7e462cfd17c4, group_id=6cb198ecbe1a4f2480de34cc9bd7e839, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:15.996110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00791c6b-f969-40fc-9a3a-32f94d20e6b1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:16.032586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 201/392] 10.222.0.78 () {58 vars in 1442 bytes} [Tue Aug 30 00:48:15 2022] PUT /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/groups/6cb198ecbe1a4f2480de34cc9bd7e839/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:48:16.095472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d603eb3-f9bf-4465-833a-3ef2dc6181fc None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:16.135321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f4380-2dd5-4409-bc12-0504ad91ba31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:16.136340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4380-2dd5-4409-bc12-0504ad91ba31 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:16.136831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4380-2dd5-4409-bc12-0504ad91ba31 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:16.137338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4380-2dd5-4409-bc12-0504ad91ba31 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:16.138700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f4380-2dd5-4409-bc12-0504ad91ba31 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:16.140360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f4380-2dd5-4409-bc12-0504ad91ba31 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:16.143357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 192/393] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:15 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:16.149492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00791c6b-f969-40fc-9a3a-32f94d20e6b1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:16.265111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00791c6b-f969-40fc-9a3a-32f94d20e6b1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:16.302691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b8888-fe1e-47c8-9a19-2b9a0b82bb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:16.303355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b8888-fe1e-47c8-9a19-2b9a0b82bb19 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:16.303624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b8888-fe1e-47c8-9a19-2b9a0b82bb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:16.303896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b8888-fe1e-47c8-9a19-2b9a0b82bb19 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/groups/6cb198ecbe1a4f2480de34cc9bd7e839/roles/26e0da7be50c47de8620dca1c657eb4f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:16.306130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b8888-fe1e-47c8-9a19-2b9a0b82bb19 None admin] RBAC: Authorizing `identity:create_grant(project_id=09da17aa18cc4293b57f7e462cfd17c4, group_id=6cb198ecbe1a4f2480de34cc9bd7e839, role_id=26e0da7be50c47de8620dca1c657eb4f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:16.307783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b8888-fe1e-47c8-9a19-2b9a0b82bb19 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:16.330120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 202/394] 10.222.0.78 () {58 vars in 1442 bytes} [Tue Aug 30 00:48:16 2022] PUT /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4/groups/6cb198ecbe1a4f2480de34cc9bd7e839/roles/26e0da7be50c47de8620dca1c657eb4f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:17.067816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c784cd-59d5-4213-9bbc-2cdcdb4e3b25 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:17.068358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c784cd-59d5-4213-9bbc-2cdcdb4e3b25 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:17.068358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c784cd-59d5-4213-9bbc-2cdcdb4e3b25 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:17.070226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 193/395] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:17.075043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2de9ad-56fe-4bec-83fc-00908711f83c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:17.075444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2de9ad-56fe-4bec-83fc-00908711f83c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:17.075611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2de9ad-56fe-4bec-83fc-00908711f83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:17.095526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7e2de9ad-56fe-4bec-83fc-00908711f83c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:17.096765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7e2de9ad-56fe-4bec-83fc-00908711f83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:17.124736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:17.196432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 203/396] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:17.396211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51665-25a0-47ec-a1b7-9d8bca9ddab5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:17.396591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51665-25a0-47ec-a1b7-9d8bca9ddab5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:17.396773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51665-25a0-47ec-a1b7-9d8bca9ddab5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:17.398372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 194/397] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:17.403411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f2a0-8a5d-4f8b-8456-9ca0f1048fac None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:17.403701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f2a0-8a5d-4f8b-8456-9ca0f1048fac None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:17.403969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f2a0-8a5d-4f8b-8456-9ca0f1048fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:17.425619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7242f2a0-8a5d-4f8b-8456-9ca0f1048fac None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:17.426791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7242f2a0-8a5d-4f8b-8456-9ca0f1048fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:17.511386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 204/398] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:17.629706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf51665-25a0-47ec-a1b7-9d8bca9ddab5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:17.724964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:17.747456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf51665-25a0-47ec-a1b7-9d8bca9ddab5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:17.788482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ae63b-7a42-4593-8d28-c069bce25173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:17.789124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ae63b-7a42-4593-8d28-c069bce25173 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:17.789406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ae63b-7a42-4593-8d28-c069bce25173 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:17.789728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ae63b-7a42-4593-8d28-c069bce25173 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:17.801198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ae63b-7a42-4593-8d28-c069bce25173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-553ae63b-7a42-4593-8d28-c069bce25173 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:17.806020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:17.806966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:17.807786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 195/399] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:17.813747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f2a0-8a5d-4f8b-8456-9ca0f1048fac None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:17.827362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ae63b-7a42-4593-8d28-c069bce25173 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:17.949579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:17.975128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f2a0-8a5d-4f8b-8456-9ca0f1048fac None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:17.975852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ae63b-7a42-4593-8d28-c069bce25173 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:18.015277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d1388-6fee-4854-bbd8-68a9ae1f08d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.016801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d1388-6fee-4854-bbd8-68a9ae1f08d8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.017263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d1388-6fee-4854-bbd8-68a9ae1f08d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.017679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d1388-6fee-4854-bbd8-68a9ae1f08d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.018478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50184e7b-a6ed-4eb1-a300-bd35259c9e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.019059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184e7b-a6ed-4eb1-a300-bd35259c9e74 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.019349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184e7b-a6ed-4eb1-a300-bd35259c9e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.019608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184e7b-a6ed-4eb1-a300-bd35259c9e74 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.020549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50184e7b-a6ed-4eb1-a300-bd35259c9e74 None admin] RBAC: Adding query filter params (name=member) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:18.020853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50184e7b-a6ed-4eb1-a300-bd35259c9e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:18.022641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50184e7b-a6ed-4eb1-a300-bd35259c9e74 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:18.030252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d1388-6fee-4854-bbd8-68a9ae1f08d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:18.033138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 205/400] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:18.034147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9401867-96a1-44de-9219-0208f58b4d6c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.034427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9401867-96a1-44de-9219-0208f58b4d6c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.034698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9401867-96a1-44de-9219-0208f58b4d6c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ac8d1388-6fee-4854-bbd8-68a9ae1f08d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:18.038000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:18.039055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:18.040060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:18.041013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 206/401] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:18.045270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 196/402] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:18.045972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9401867-96a1-44de-9219-0208f58b4d6c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:18.049568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d1388-6fee-4854-bbd8-68a9ae1f08d8 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:18.054441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:18.168488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9401867-96a1-44de-9219-0208f58b4d6c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:18.183712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d1388-6fee-4854-bbd8-68a9ae1f08d8 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:18.215755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf34af-bdc8-473e-b354-12836048cb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.216395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf34af-bdc8-473e-b354-12836048cb64 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.217113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf34af-bdc8-473e-b354-12836048cb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.217398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf34af-bdc8-473e-b354-12836048cb64 None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.219813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf34af-bdc8-473e-b354-12836048cb64 None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:18.221287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf34af-bdc8-473e-b354-12836048cb64 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:18.223074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 207/403] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:18 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:18.223698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c92ced-e774-4b63-8a74-0fc088e20914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.224612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c92ced-e774-4b63-8a74-0fc088e20914 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.225186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c92ced-e774-4b63-8a74-0fc088e20914 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.225656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c92ced-e774-4b63-8a74-0fc088e20914 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.226534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c92ced-e774-4b63-8a74-0fc088e20914 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:18.227045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c92ced-e774-4b63-8a74-0fc088e20914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:18.228694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaaed6-650c-43f8-bda0-0a52cb0d5871 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.228972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaaed6-650c-43f8-bda0-0a52cb0d5871 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.229265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaaed6-650c-43f8-bda0-0a52cb0d5871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.229640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c92ced-e774-4b63-8a74-0fc088e20914 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:18.240152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 197/404] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:18.241335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c92ced-e774-4b63-8a74-0fc088e20914 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:18.256721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0beaaed6-650c-43f8-bda0-0a52cb0d5871 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:18.258954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0beaaed6-650c-43f8-bda0-0a52cb0d5871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:18.268684 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:18.334338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 208/405] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:18.335603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaaed6-650c-43f8-bda0-0a52cb0d5871 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:18.337149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:18.381574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c92ced-e774-4b63-8a74-0fc088e20914 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:18.424329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86596c1-75c2-4625-869a-6d17950cf5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.425069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86596c1-75c2-4625-869a-6d17950cf5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.425373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86596c1-75c2-4625-869a-6d17950cf5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.425645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86596c1-75c2-4625-869a-6d17950cf5d9 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.426792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86596c1-75c2-4625-869a-6d17950cf5d9 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:18.428821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86596c1-75c2-4625-869a-6d17950cf5d9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:18.431001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 198/406] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:18 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:18.439647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86596c1-75c2-4625-869a-6d17950cf5d9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:18.466869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaaed6-650c-43f8-bda0-0a52cb0d5871 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:18.535840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cf743-e992-4d41-84ec-35a68e7643a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.536812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cf743-e992-4d41-84ec-35a68e7643a1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.537381 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cf743-e992-4d41-84ec-35a68e7643a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.537854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cf743-e992-4d41-84ec-35a68e7643a1 None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.544367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cf743-e992-4d41-84ec-35a68e7643a1 None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:18.546177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cf743-e992-4d41-84ec-35a68e7643a1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:18.548968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 209/407] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:18 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:18.554188 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86596c1-75c2-4625-869a-6d17950cf5d9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:18.562397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cf743-e992-4d41-84ec-35a68e7643a1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:18.589282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:18.592835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ec00f-fe2c-48cb-951b-d10963db7a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.592835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ec00f-fe2c-48cb-951b-d10963db7a57 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.592835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ec00f-fe2c-48cb-951b-d10963db7a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.592835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ec00f-fe2c-48cb-951b-d10963db7a57 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932/groups/6cb198ecbe1a4f2480de34cc9bd7e839/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.596614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ec00f-fe2c-48cb-951b-d10963db7a57 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5a0684693f546a9b65d6dd77f36a932, group_id=6cb198ecbe1a4f2480de34cc9bd7e839, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:18.596614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ec00f-fe2c-48cb-951b-d10963db7a57 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:18.601445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:18.627521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 199/408] 10.222.0.78 () {58 vars in 1442 bytes} [Tue Aug 30 00:48:18 2022] PUT /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932/groups/6cb198ecbe1a4f2480de34cc9bd7e839/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:18.667088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ec00f-fe2c-48cb-951b-d10963db7a57 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:18.710611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cf743-e992-4d41-84ec-35a68e7643a1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:18.758120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaddd21-645c-4ec7-9606-3bd8f6bf94c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.758770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaddd21-645c-4ec7-9606-3bd8f6bf94c2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.759051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaddd21-645c-4ec7-9606-3bd8f6bf94c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.759325 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaddd21-645c-4ec7-9606-3bd8f6bf94c2 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.761834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaddd21-645c-4ec7-9606-3bd8f6bf94c2 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:18.763587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaddd21-645c-4ec7-9606-3bd8f6bf94c2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:18.766925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 210/409] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:18 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:18.768436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8b798-9b3c-476a-8b23-9be11eddb49b None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.768926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8b798-9b3c-476a-8b23-9be11eddb49b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.769446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8b798-9b3c-476a-8b23-9be11eddb49b None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.776168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 211/410] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:18.777506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8b798-9b3c-476a-8b23-9be11eddb49b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:18.812428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ec00f-fe2c-48cb-951b-d10963db7a57 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:18.852546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217996f5-36d8-45b7-a270-536e7969a4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.853198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217996f5-36d8-45b7-a270-536e7969a4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.853528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217996f5-36d8-45b7-a270-536e7969a4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.853741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217996f5-36d8-45b7-a270-536e7969a4a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.864498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217996f5-36d8-45b7-a270-536e7969a4a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-217996f5-36d8-45b7-a270-536e7969a4a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:18.869715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:18.870613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 200/411] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:18.874027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32298fbf-2d97-4177-9567-6c1f075973f7 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.874737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32298fbf-2d97-4177-9567-6c1f075973f7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.874737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32298fbf-2d97-4177-9567-6c1f075973f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.884460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:18.892353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-32298fbf-2d97-4177-9567-6c1f075973f7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:18.893575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-32298fbf-2d97-4177-9567-6c1f075973f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:18.911346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8b798-9b3c-476a-8b23-9be11eddb49b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:18.947506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:18.949651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e98c3a-2d85-4504-972b-417bb74c8912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:18.950335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e98c3a-2d85-4504-972b-417bb74c8912 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.950611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e98c3a-2d85-4504-972b-417bb74c8912 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.950923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e98c3a-2d85-4504-972b-417bb74c8912 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932/groups/6cb198ecbe1a4f2480de34cc9bd7e839/roles/26e0da7be50c47de8620dca1c657eb4f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.953226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e98c3a-2d85-4504-972b-417bb74c8912 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5a0684693f546a9b65d6dd77f36a932, group_id=6cb198ecbe1a4f2480de34cc9bd7e839, role_id=26e0da7be50c47de8620dca1c657eb4f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:18.955522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e98c3a-2d85-4504-972b-417bb74c8912 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:18.969130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 201/412] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:18.970219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec761a09-6af5-4db2-bdcf-0e946812651f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.970499 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec761a09-6af5-4db2-bdcf-0e946812651f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.970767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec761a09-6af5-4db2-bdcf-0e946812651f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.976182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 202/413] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:18.976182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec761a09-6af5-4db2-bdcf-0e946812651f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:18.976399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 212/414] 10.222.0.78 () {58 vars in 1442 bytes} [Tue Aug 30 00:48:18 2022] PUT /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932/groups/6cb198ecbe1a4f2480de34cc9bd7e839/roles/26e0da7be50c47de8620dca1c657eb4f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:18.982646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8e4fa-5831-41cf-a7d3-2f67aee084c1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.982938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8e4fa-5831-41cf-a7d3-2f67aee084c1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.983195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8e4fa-5831-41cf-a7d3-2f67aee084c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.033188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b4f8e4fa-5831-41cf-a7d3-2f67aee084c1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:19.034392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b4f8e4fa-5831-41cf-a7d3-2f67aee084c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:19.096904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec761a09-6af5-4db2-bdcf-0e946812651f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.136756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91a4dc-a70d-485f-aed3-5f6c61f42048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.137398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91a4dc-a70d-485f-aed3-5f6c61f42048 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.137673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91a4dc-a70d-485f-aed3-5f6c61f42048 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.137940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91a4dc-a70d-485f-aed3-5f6c61f42048 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.138609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91a4dc-a70d-485f-aed3-5f6c61f42048 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:19.138908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91a4dc-a70d-485f-aed3-5f6c61f42048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:19.140290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91a4dc-a70d-485f-aed3-5f6c61f42048 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:19.151092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 203/415] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:19.157437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91a4dc-a70d-485f-aed3-5f6c61f42048 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.202218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 213/416] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:19.219971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.282363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91a4dc-a70d-485f-aed3-5f6c61f42048 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.328201 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea2ce-4121-47e2-858b-c81a0e0bfb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.329250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea2ce-4121-47e2-858b-c81a0e0bfb1f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.329766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea2ce-4121-47e2-858b-c81a0e0bfb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.330231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea2ce-4121-47e2-858b-c81a0e0bfb1f None admin] PATH_INFO: `/v3/groups/b7dc9b6aaf524b42b6a12f628ed42be6` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.331596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ea2ce-4121-47e2-858b-c81a0e0bfb1f None admin] RBAC: Authorizing `identity:get_group(group_id=b7dc9b6aaf524b42b6a12f628ed42be6)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:19.333297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ea2ce-4121-47e2-858b-c81a0e0bfb1f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:19.335059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 204/417] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/groups/b7dc9b6aaf524b42b6a12f628ed42be6 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:19.340444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8e4fa-5831-41cf-a7d3-2f67aee084c1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.363486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea2ce-4121-47e2-858b-c81a0e0bfb1f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.476864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.537598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8e4fa-5831-41cf-a7d3-2f67aee084c1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.551375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea2ce-4121-47e2-858b-c81a0e0bfb1f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.576776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.597803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a87a61-67ff-4a9b-be08-449602202fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.600608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a87a61-67ff-4a9b-be08-449602202fdd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.601627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a87a61-67ff-4a9b-be08-449602202fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.601902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a87a61-67ff-4a9b-be08-449602202fdd None admin] PATH_INFO: `/v3/projects/761e332e13aa43bb851c08b247de0d02` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.603145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a87a61-67ff-4a9b-be08-449602202fdd None admin] RBAC: Authorizing `identity:get_project(project_id=761e332e13aa43bb851c08b247de0d02)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:19.604576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a87a61-67ff-4a9b-be08-449602202fdd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:19.609227 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 214/418] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/projects/761e332e13aa43bb851c08b247de0d02 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:19.610865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a87a61-67ff-4a9b-be08-449602202fdd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.615346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21df535-319c-4003-a661-a7510f4c02bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.615926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21df535-319c-4003-a661-a7510f4c02bb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.616208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21df535-319c-4003-a661-a7510f4c02bb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.616481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21df535-319c-4003-a661-a7510f4c02bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.627651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21df535-319c-4003-a661-a7510f4c02bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-e21df535-319c-4003-a661-a7510f4c02bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:19.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:19.633887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:19.634682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 205/419] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:19.635324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21df535-319c-4003-a661-a7510f4c02bb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.747331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a87a61-67ff-4a9b-be08-449602202fdd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.751701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21df535-319c-4003-a661-a7510f4c02bb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.790652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98562902-7038-46ea-a981-6d5f2dbf2797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.791325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98562902-7038-46ea-a981-6d5f2dbf2797 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.791590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98562902-7038-46ea-a981-6d5f2dbf2797 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.791868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98562902-7038-46ea-a981-6d5f2dbf2797 None admin] PATH_INFO: `/v3/projects/761e332e13aa43bb851c08b247de0d02/groups/b7dc9b6aaf524b42b6a12f628ed42be6/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.792601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffafaa5-6a8e-423c-ade0-2be1c69f072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.793375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffafaa5-6a8e-423c-ade0-2be1c69f072c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.793810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffafaa5-6a8e-423c-ade0-2be1c69f072c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.794172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffafaa5-6a8e-423c-ade0-2be1c69f072c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.794492 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98562902-7038-46ea-a981-6d5f2dbf2797 None admin] RBAC: Authorizing `identity:create_grant(project_id=761e332e13aa43bb851c08b247de0d02, group_id=b7dc9b6aaf524b42b6a12f628ed42be6, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:19.796076 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98562902-7038-46ea-a981-6d5f2dbf2797 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:19.800850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.808349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffafaa5-6a8e-423c-ade0-2be1c69f072c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:19.812698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 206/420] 10.222.0.78 () {58 vars in 1442 bytes} [Tue Aug 30 00:48:19 2022] PUT /identity/v3/projects/761e332e13aa43bb851c08b247de0d02/groups/b7dc9b6aaf524b42b6a12f628ed42be6/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:19.814340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98562902-7038-46ea-a981-6d5f2dbf2797 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-7ffafaa5-6a8e-423c-ade0-2be1c69f072c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:19.824270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:19.825463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:19.826469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:19.830710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 215/421] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:19.837667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffafaa5-6a8e-423c-ade0-2be1c69f072c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.899624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.991722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffafaa5-6a8e-423c-ade0-2be1c69f072c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.023588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98562902-7038-46ea-a981-6d5f2dbf2797 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.027280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f48750-452e-434b-967a-94591371bc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.027898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f48750-452e-434b-967a-94591371bc54 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.028181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f48750-452e-434b-967a-94591371bc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.028451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f48750-452e-434b-967a-94591371bc54 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.029663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f48750-452e-434b-967a-94591371bc54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:20.029663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f48750-452e-434b-967a-94591371bc54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.030823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f48750-452e-434b-967a-94591371bc54 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.040264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 216/422] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:20.048299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f48750-452e-434b-967a-94591371bc54 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.067868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b91cd-9efc-4d12-88db-3d74bb159a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.069729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b91cd-9efc-4d12-88db-3d74bb159a08 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.070218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b91cd-9efc-4d12-88db-3d74bb159a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.070682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b91cd-9efc-4d12-88db-3d74bb159a08 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.071532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b91cd-9efc-4d12-88db-3d74bb159a08 None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:20.072020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b91cd-9efc-4d12-88db-3d74bb159a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.073595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b91cd-9efc-4d12-88db-3d74bb159a08 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.085209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 207/423] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:20.090448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b91cd-9efc-4d12-88db-3d74bb159a08 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.124714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.187568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f48750-452e-434b-967a-94591371bc54 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.239532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.252331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b91cd-9efc-4d12-88db-3d74bb159a08 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.258527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92510b41-3d45-4d10-8a28-b40e972f36ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.259188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510b41-3d45-4d10-8a28-b40e972f36ce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.259474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510b41-3d45-4d10-8a28-b40e972f36ce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.259745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510b41-3d45-4d10-8a28-b40e972f36ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.300914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770d335-3b0e-4940-bb44-1d71414a7133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.301566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d335-3b0e-4940-bb44-1d71414a7133 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.301849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d335-3b0e-4940-bb44-1d71414a7133 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.302122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d335-3b0e-4940-bb44-1d71414a7133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.373621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770d335-3b0e-4940-bb44-1d71414a7133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.375147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770d335-3b0e-4940-bb44-1d71414a7133 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.385274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92510b41-3d45-4d10-8a28-b40e972f36ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.386840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92510b41-3d45-4d10-8a28-b40e972f36ce None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.494756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.521915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.536893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 208/424] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:20.543710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770d335-3b0e-4940-bb44-1d71414a7133 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.575753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 217/425] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:20.581992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92510b41-3d45-4d10-8a28-b40e972f36ce None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.651352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770d335-3b0e-4940-bb44-1d71414a7133 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.684717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92510b41-3d45-4d10-8a28-b40e972f36ce None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.718235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d26554-2109-4666-9c73-3e4c48ad8aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.718924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d26554-2109-4666-9c73-3e4c48ad8aab None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.719767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d26554-2109-4666-9c73-3e4c48ad8aab None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.720031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d26554-2109-4666-9c73-3e4c48ad8aab None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.722095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb09da-d41e-458b-8b08-648001e01c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.722942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb09da-d41e-458b-8b08-648001e01c62 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.723433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb09da-d41e-458b-8b08-648001e01c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.723909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb09da-d41e-458b-8b08-648001e01c62 None admin] PATH_INFO: `/v3/projects/09da17aa18cc4293b57f7e462cfd17c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.725272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb09da-d41e-458b-8b08-648001e01c62 None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.725781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d26554-2109-4666-9c73-3e4c48ad8aab None admin] RBAC: Authorizing `identity:get_project(project_id=09da17aa18cc4293b57f7e462cfd17c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.727148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb09da-d41e-458b-8b08-648001e01c62 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.727256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d26554-2109-4666-9c73-3e4c48ad8aab None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.729043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 209/426] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:20.729926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 218/427] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/projects/09da17aa18cc4293b57f7e462cfd17c4 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:20.734389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d26554-2109-4666-9c73-3e4c48ad8aab None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.736598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb09da-d41e-458b-8b08-648001e01c62 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.764919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.841345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.842695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb09da-d41e-458b-8b08-648001e01c62 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.878741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb6e28-6a7f-42cb-b851-8be33dfe3808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.879721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb6e28-6a7f-42cb-b851-8be33dfe3808 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.880310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb6e28-6a7f-42cb-b851-8be33dfe3808 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.880774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb6e28-6a7f-42cb-b851-8be33dfe3808 None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.882194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb6e28-6a7f-42cb-b851-8be33dfe3808 None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.883861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb6e28-6a7f-42cb-b851-8be33dfe3808 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.883967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d26554-2109-4666-9c73-3e4c48ad8aab None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.885862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 219/428] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:20.891716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b44b1-1046-4c9c-ad5a-081e5bcc8960 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.892408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b44b1-1046-4c9c-ad5a-081e5bcc8960 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.892904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b44b1-1046-4c9c-ad5a-081e5bcc8960 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.894785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 220/429] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:20.896708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b44b1-1046-4c9c-ad5a-081e5bcc8960 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.921196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c48aa-1216-4405-a094-ae0c0096a21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.921612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c48aa-1216-4405-a094-ae0c0096a21c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.921885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c48aa-1216-4405-a094-ae0c0096a21c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.922185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c48aa-1216-4405-a094-ae0c0096a21c None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.923766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c48aa-1216-4405-a094-ae0c0096a21c None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.925356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c48aa-1216-4405-a094-ae0c0096a21c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.927604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 210/430] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:20.929458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876bcec-ccf4-4840-a96f-786d4e971380 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.929673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876bcec-ccf4-4840-a96f-786d4e971380 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.929944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876bcec-ccf4-4840-a96f-786d4e971380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.950186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f876bcec-ccf4-4840-a96f-786d4e971380 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:20.951529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f876bcec-ccf4-4840-a96f-786d4e971380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:21.001628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.010528 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b44b1-1046-4c9c-ad5a-081e5bcc8960 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.026504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 211/431] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:21.027369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876bcec-ccf4-4840-a96f-786d4e971380 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.051065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaaff9-a8be-425a-946b-69dad3a028c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.052092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaaff9-a8be-425a-946b-69dad3a028c9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.052566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaaff9-a8be-425a-946b-69dad3a028c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.053027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaaff9-a8be-425a-946b-69dad3a028c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.054075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaaff9-a8be-425a-946b-69dad3a028c9 None admin] RBAC: Adding query filter params (group.id=6cb198ecbe1a4f2480de34cc9bd7e839, role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:21.054594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaaff9-a8be-425a-946b-69dad3a028c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.056295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaaff9-a8be-425a-946b-69dad3a028c9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:21.063451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 221/432] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/role_assignments?group.id=6cb198ecbe1a4f2480de34cc9bd7e839&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:21.100046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.144976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876bcec-ccf4-4840-a96f-786d4e971380 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.185767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0ed80-7812-44ff-a072-121c5682faa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.186379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0ed80-7812-44ff-a072-121c5682faa9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.186655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0ed80-7812-44ff-a072-121c5682faa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.186925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0ed80-7812-44ff-a072-121c5682faa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.187662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0ed80-7812-44ff-a072-121c5682faa9 None admin] RBAC: Adding query filter params (group.id=6cb198ecbe1a4f2480de34cc9bd7e839, role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=09da17aa18cc4293b57f7e462cfd17c4) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:21.188641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0ed80-7812-44ff-a072-121c5682faa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.190216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0ed80-7812-44ff-a072-121c5682faa9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:21.197553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 212/433] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/role_assignments?group.id=6cb198ecbe1a4f2480de34cc9bd7e839&scope.project.id=09da17aa18cc4293b57f7e462cfd17c4&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:21.202718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33be8a0-c49e-41e6-b588-bea9e018d199 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.203044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33be8a0-c49e-41e6-b588-bea9e018d199 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.203267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33be8a0-c49e-41e6-b588-bea9e018d199 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.210092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 222/434] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:21.211991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae163345-d02d-4077-83cf-e93d70dcbd7e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.212269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae163345-d02d-4077-83cf-e93d70dcbd7e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.212537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae163345-d02d-4077-83cf-e93d70dcbd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.240772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ae163345-d02d-4077-83cf-e93d70dcbd7e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:21.242250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ae163345-d02d-4077-83cf-e93d70dcbd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:21.326382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 213/435] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:21.410239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33be8a0-c49e-41e6-b588-bea9e018d199 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.525501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33be8a0-c49e-41e6-b588-bea9e018d199 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.561306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521f80d-0c75-451b-9c89-9164716c111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.562232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521f80d-0c75-451b-9c89-9164716c111e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.562678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521f80d-0c75-451b-9c89-9164716c111e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.563273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521f80d-0c75-451b-9c89-9164716c111e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.574126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521f80d-0c75-451b-9c89-9164716c111e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-9521f80d-0c75-451b-9c89-9164716c111e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:21.579862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:21.580726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:21.581532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:21.583670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 223/436] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:21.589253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae163345-d02d-4077-83cf-e93d70dcbd7e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.630137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521f80d-0c75-451b-9c89-9164716c111e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.656314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.659802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.721484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae163345-d02d-4077-83cf-e93d70dcbd7e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.761579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4f43b-bfcc-4000-995a-4141a35d809a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.764013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4f43b-bfcc-4000-995a-4141a35d809a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.764013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4f43b-bfcc-4000-995a-4141a35d809a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.764013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4f43b-bfcc-4000-995a-4141a35d809a None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.764013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4f43b-bfcc-4000-995a-4141a35d809a None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:21.764013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4f43b-bfcc-4000-995a-4141a35d809a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.765032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4f43b-bfcc-4000-995a-4141a35d809a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:21.769557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521f80d-0c75-451b-9c89-9164716c111e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.774760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 214/437] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:21.780254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4f43b-bfcc-4000-995a-4141a35d809a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.817523 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93eb31-a47c-4d18-b14e-cc044b8b4733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.818516 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93eb31-a47c-4d18-b14e-cc044b8b4733 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.818964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93eb31-a47c-4d18-b14e-cc044b8b4733 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.819809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93eb31-a47c-4d18-b14e-cc044b8b4733 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.835873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93eb31-a47c-4d18-b14e-cc044b8b4733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-0a93eb31-a47c-4d18-b14e-cc044b8b4733 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:21.842031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:21.842914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:21.843694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:21.844884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 224/438] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:21.851000 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93eb31-a47c-4d18-b14e-cc044b8b4733 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.908829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4f43b-bfcc-4000-995a-4141a35d809a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.931539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.955623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31b556-d676-4e52-bcbf-252f059049e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.956231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31b556-d676-4e52-bcbf-252f059049e0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.956505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31b556-d676-4e52-bcbf-252f059049e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.956767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31b556-d676-4e52-bcbf-252f059049e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.961868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.989380 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93eb31-a47c-4d18-b14e-cc044b8b4733 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.026666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8ebbe-5c04-4c4a-8b8d-2ac4b83e8e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.027272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8ebbe-5c04-4c4a-8b8d-2ac4b83e8e99 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.027545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8ebbe-5c04-4c4a-8b8d-2ac4b83e8e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.027811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8ebbe-5c04-4c4a-8b8d-2ac4b83e8e99 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.028468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8ebbe-5c04-4c4a-8b8d-2ac4b83e8e99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:22.028773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8ebbe-5c04-4c4a-8b8d-2ac4b83e8e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.030192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8ebbe-5c04-4c4a-8b8d-2ac4b83e8e99 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.036683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31b556-d676-4e52-bcbf-252f059049e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.038136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31b556-d676-4e52-bcbf-252f059049e0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.038898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 225/439] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:22.044229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8ebbe-5c04-4c4a-8b8d-2ac4b83e8e99 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.146673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8ebbe-5c04-4c4a-8b8d-2ac4b83e8e99 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.147906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 215/440] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:22.153347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31b556-d676-4e52-bcbf-252f059049e0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.171194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.181417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de786560-a436-4af2-aa3a-2bf4091a40e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.182020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de786560-a436-4af2-aa3a-2bf4091a40e0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.182304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de786560-a436-4af2-aa3a-2bf4091a40e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.182577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de786560-a436-4af2-aa3a-2bf4091a40e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.195516 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.266965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31b556-d676-4e52-bcbf-252f059049e0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.268308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de786560-a436-4af2-aa3a-2bf4091a40e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.269938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de786560-a436-4af2-aa3a-2bf4091a40e0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.302962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58504e0-2367-48ba-bd99-a5a77a7bb81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.303584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58504e0-2367-48ba-bd99-a5a77a7bb81e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.303868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58504e0-2367-48ba-bd99-a5a77a7bb81e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.304140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58504e0-2367-48ba-bd99-a5a77a7bb81e None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.305298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58504e0-2367-48ba-bd99-a5a77a7bb81e None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.306714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58504e0-2367-48ba-bd99-a5a77a7bb81e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.308484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 216/441] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:22.309616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d039c-a4c6-4ea9-942a-69051c03fc2f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.309893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d039c-a4c6-4ea9-942a-69051c03fc2f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.310162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d039c-a4c6-4ea9-942a-69051c03fc2f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.311345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 217/442] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:22.314952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d039c-a4c6-4ea9-942a-69051c03fc2f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.374445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 226/443] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:48:22.376527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d6fe3-7f5a-4ce8-b116-4013d441ba3a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.376813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d6fe3-7f5a-4ce8-b116-4013d441ba3a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.377099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d6fe3-7f5a-4ce8-b116-4013d441ba3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.394100 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d80d6fe3-7f5a-4ce8-b116-4013d441ba3a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:22.395290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d80d6fe3-7f5a-4ce8-b116-4013d441ba3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:22.406428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.423762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d039c-a4c6-4ea9-942a-69051c03fc2f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.429867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.459095 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c532cc-b3b4-44cf-9743-f51f0e03f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.459723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c532cc-b3b4-44cf-9743-f51f0e03f606 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.460005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c532cc-b3b4-44cf-9743-f51f0e03f606 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.460280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c532cc-b3b4-44cf-9743-f51f0e03f606 None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.461435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c532cc-b3b4-44cf-9743-f51f0e03f606 None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.462859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c532cc-b3b4-44cf-9743-f51f0e03f606 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.463220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 227/444] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:22.464045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d6fe3-7f5a-4ce8-b116-4013d441ba3a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.464622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 218/445] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:22.470271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c532cc-b3b4-44cf-9743-f51f0e03f606 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.573488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d6fe3-7f5a-4ce8-b116-4013d441ba3a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.573834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c532cc-b3b4-44cf-9743-f51f0e03f606 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.608481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be2214-b691-4bb2-af2c-df6e50688b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.608663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b82d4-7260-45a9-8d4b-59bd575c9877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.609129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b82d4-7260-45a9-8d4b-59bd575c9877 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.609232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be2214-b691-4bb2-af2c-df6e50688b8b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.609413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b82d4-7260-45a9-8d4b-59bd575c9877 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.609508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be2214-b691-4bb2-af2c-df6e50688b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.610232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be2214-b691-4bb2-af2c-df6e50688b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.610336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b82d4-7260-45a9-8d4b-59bd575c9877 None admin] PATH_INFO: `/v3/projects/b5a0684693f546a9b65d6dd77f36a932` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.611108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be2214-b691-4bb2-af2c-df6e50688b8b None admin] RBAC: Adding query filter params (group.id=6cb198ecbe1a4f2480de34cc9bd7e839, role.id=e407aec03c22425591fa0e52a40edbc1, scope.project.id=b5a0684693f546a9b65d6dd77f36a932) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:22.611415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be2214-b691-4bb2-af2c-df6e50688b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.611506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b82d4-7260-45a9-8d4b-59bd575c9877 None admin] RBAC: Authorizing `identity:get_project(project_id=b5a0684693f546a9b65d6dd77f36a932)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.612834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be2214-b691-4bb2-af2c-df6e50688b8b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.612942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b82d4-7260-45a9-8d4b-59bd575c9877 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.619011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 228/446] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/projects/b5a0684693f546a9b65d6dd77f36a932 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:22.619992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b82d4-7260-45a9-8d4b-59bd575c9877 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.629204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 219/447] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/role_assignments?group.id=6cb198ecbe1a4f2480de34cc9bd7e839&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=e407aec03c22425591fa0e52a40edbc1 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:22.673289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.740630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b82d4-7260-45a9-8d4b-59bd575c9877 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.745480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be2214-b691-4bb2-af2c-df6e50688b8b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.770719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.780767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549eeb5-5747-44cd-8729-54db36996ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.782396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549eeb5-5747-44cd-8729-54db36996ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.782545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549eeb5-5747-44cd-8729-54db36996ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.782805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549eeb5-5747-44cd-8729-54db36996ce3 None admin] PATH_INFO: `/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.784058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549eeb5-5747-44cd-8729-54db36996ce3 None admin] RBAC: Authorizing `identity:get_group(group_id=6cb198ecbe1a4f2480de34cc9bd7e839)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.785555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549eeb5-5747-44cd-8729-54db36996ce3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.789120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 229/448] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/groups/6cb198ecbe1a4f2480de34cc9bd7e839 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:22.796227 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549eeb5-5747-44cd-8729-54db36996ce3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.873932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be2214-b691-4bb2-af2c-df6e50688b8b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.917693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa43a8-65a5-4edd-be1d-902176c3ebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.918424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa43a8-65a5-4edd-be1d-902176c3ebaa None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.918768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa43a8-65a5-4edd-be1d-902176c3ebaa None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.919107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa43a8-65a5-4edd-be1d-902176c3ebaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.930409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549eeb5-5747-44cd-8729-54db36996ce3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.931349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa43a8-65a5-4edd-be1d-902176c3ebaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3aaa43a8-65a5-4edd-be1d-902176c3ebaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:22.936759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:22.937667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:22.939257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:22.939847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 220/449] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:22.943261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.945712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa43a8-65a5-4edd-be1d-902176c3ebaa None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.974073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102502e-a5d1-47d2-a268-d7df03c1a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.974681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102502e-a5d1-47d2-a268-d7df03c1a010 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.974942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102502e-a5d1-47d2-a268-d7df03c1a010 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.975208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102502e-a5d1-47d2-a268-d7df03c1a010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.975928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102502e-a5d1-47d2-a268-d7df03c1a010 None admin] RBAC: Adding query filter params (group.id=6cb198ecbe1a4f2480de34cc9bd7e839, role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=b5a0684693f546a9b65d6dd77f36a932) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:22.976238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102502e-a5d1-47d2-a268-d7df03c1a010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.977711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102502e-a5d1-47d2-a268-d7df03c1a010 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.983892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 230/450] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/role_assignments?group.id=6cb198ecbe1a4f2480de34cc9bd7e839&scope.project.id=b5a0684693f546a9b65d6dd77f36a932&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:23.042092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:23.067770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa43a8-65a5-4edd-be1d-902176c3ebaa None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:23.103242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fc279-fc14-4678-a522-cd50c1481cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:23.103858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fc279-fc14-4678-a522-cd50c1481cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:23.104140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fc279-fc14-4678-a522-cd50c1481cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:23.104413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fc279-fc14-4678-a522-cd50c1481cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:23.105105 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fc279-fc14-4678-a522-cd50c1481cb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:23.105418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fc279-fc14-4678-a522-cd50c1481cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:23.106847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fc279-fc14-4678-a522-cd50c1481cb6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:23.116722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 221/451] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:23.122414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102502e-a5d1-47d2-a268-d7df03c1a010 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:23.228766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102502e-a5d1-47d2-a268-d7df03c1a010 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:23.267137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf4c98-9b8a-400e-ae80-b073477b0670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:23.267747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf4c98-9b8a-400e-ae80-b073477b0670 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:23.268027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf4c98-9b8a-400e-ae80-b073477b0670 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:23.268299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf4c98-9b8a-400e-ae80-b073477b0670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:23.307487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:23.336767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf4c98-9b8a-400e-ae80-b073477b0670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:23.338240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf4c98-9b8a-400e-ae80-b073477b0670 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:23.432008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 231/452] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:23.438075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fc279-fc14-4678-a522-cd50c1481cb6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:23.535179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fc279-fc14-4678-a522-cd50c1481cb6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:23.567175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb90a3-f663-4b7c-9c47-fe8a4d86de36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:23.567785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb90a3-f663-4b7c-9c47-fe8a4d86de36 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:23.568064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb90a3-f663-4b7c-9c47-fe8a4d86de36 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:23.568332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb90a3-f663-4b7c-9c47-fe8a4d86de36 None admin] PATH_INFO: `/v3/projects/761e332e13aa43bb851c08b247de0d02` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:23.569718 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb90a3-f663-4b7c-9c47-fe8a4d86de36 None admin] RBAC: Authorizing `identity:get_project(project_id=761e332e13aa43bb851c08b247de0d02)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:23.571133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb90a3-f663-4b7c-9c47-fe8a4d86de36 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:23.572738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 222/453] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:23 2022] GET /identity/v3/projects/761e332e13aa43bb851c08b247de0d02 => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:23.579435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf4c98-9b8a-400e-ae80-b073477b0670 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:23.644612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:23.675075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf4c98-9b8a-400e-ae80-b073477b0670 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:23.707579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e2519-7140-4ac0-8f31-a629970ff3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:23.708205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e2519-7140-4ac0-8f31-a629970ff3fe None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:23.708488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e2519-7140-4ac0-8f31-a629970ff3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:23.708760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e2519-7140-4ac0-8f31-a629970ff3fe None admin] PATH_INFO: `/v3/groups/b7dc9b6aaf524b42b6a12f628ed42be6` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:23.709926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e2519-7140-4ac0-8f31-a629970ff3fe None admin] RBAC: Authorizing `identity:get_group(group_id=b7dc9b6aaf524b42b6a12f628ed42be6)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:23.711350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e2519-7140-4ac0-8f31-a629970ff3fe None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:23.712909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 232/454] 10.222.0.78 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:23 2022] GET /identity/v3/groups/b7dc9b6aaf524b42b6a12f628ed42be6 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:23.718714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb90a3-f663-4b7c-9c47-fe8a4d86de36 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:23.730489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:23.817683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb90a3-f663-4b7c-9c47-fe8a4d86de36 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:23.851502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e732faa-50cf-44be-b527-8cfab24136ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:23.852089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e732faa-50cf-44be-b527-8cfab24136ce None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:23.852392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e732faa-50cf-44be-b527-8cfab24136ce None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:23.852642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e732faa-50cf-44be-b527-8cfab24136ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:23.853394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e732faa-50cf-44be-b527-8cfab24136ce None admin] RBAC: Adding query filter params (group.id=b7dc9b6aaf524b42b6a12f628ed42be6, role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=761e332e13aa43bb851c08b247de0d02) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:23.853702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e732faa-50cf-44be-b527-8cfab24136ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:23.855101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e732faa-50cf-44be-b527-8cfab24136ce None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:23.860644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 223/455] 10.222.0.78 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:23 2022] GET /identity/v3/role_assignments?group.id=b7dc9b6aaf524b42b6a12f628ed42be6&scope.project.id=761e332e13aa43bb851c08b247de0d02&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:26.106068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ffcd-a657-4581-a90e-1da27024445f None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.106578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ffcd-a657-4581-a90e-1da27024445f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.106578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ffcd-a657-4581-a90e-1da27024445f None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.107940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 233/456] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:26.114231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df63857-212b-4561-ab87-c54be2f2fbaf None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.114676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df63857-212b-4561-ab87-c54be2f2fbaf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.114874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df63857-212b-4561-ab87-c54be2f2fbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.136665 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9df63857-212b-4561-ab87-c54be2f2fbaf None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:26.137882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9df63857-212b-4561-ab87-c54be2f2fbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:26.236398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 224/457] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:26.523204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc1c36-0786-4cfd-932e-d5ce5f3d95af None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.523880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc1c36-0786-4cfd-932e-d5ce5f3d95af None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.524358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc1c36-0786-4cfd-932e-d5ce5f3d95af None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.525938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 234/458] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:26.531930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76e69-c990-4cfb-879d-78197d854954 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.532493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76e69-c990-4cfb-879d-78197d854954 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.533064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76e69-c990-4cfb-879d-78197d854954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.542720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc1c36-0786-4cfd-932e-d5ce5f3d95af None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.545205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:26.562162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c1c76e69-c990-4cfb-879d-78197d854954 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:26.563508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c1c76e69-c990-4cfb-879d-78197d854954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:26.684638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 225/459] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:26.710325 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc1c36-0786-4cfd-932e-d5ce5f3d95af None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.728304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8135b-9ad4-4fce-8ed1-61539834aa77 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.728583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8135b-9ad4-4fce-8ed1-61539834aa77 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.729046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8135b-9ad4-4fce-8ed1-61539834aa77 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.731134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 226/460] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:26.736394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f409-a35a-4a36-b91a-b12f02414123 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.737023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f409-a35a-4a36-b91a-b12f02414123 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.737941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f409-a35a-4a36-b91a-b12f02414123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.770181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2741f409-a35a-4a36-b91a-b12f02414123 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:26.775930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2741f409-a35a-4a36-b91a-b12f02414123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:26.787362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49d088-04da-41df-a934-9777b086053a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.788267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49d088-04da-41df-a934-9777b086053a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.788757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49d088-04da-41df-a934-9777b086053a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.789246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49d088-04da-41df-a934-9777b086053a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.801451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:26.810089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49d088-04da-41df-a934-9777b086053a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:26.811542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49d088-04da-41df-a934-9777b086053a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-7c49d088-04da-41df-a934-9777b086053a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:26.833660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:26.834790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:26.835831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:26.835831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:26.835831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:26.837689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 235/461] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:26.843252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49d088-04da-41df-a934-9777b086053a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.916443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 227/462] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:26.916443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1847d-9e56-48d7-9f1c-191bc9d9badf None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.916443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1847d-9e56-48d7-9f1c-191bc9d9badf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.916443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1847d-9e56-48d7-9f1c-191bc9d9badf None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.916443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 228/463] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:26.923852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6350379-a0aa-4982-ac9e-7476bbe3fb11 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.925749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6350379-a0aa-4982-ac9e-7476bbe3fb11 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.925749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6350379-a0aa-4982-ac9e-7476bbe3fb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.966995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c6350379-a0aa-4982-ac9e-7476bbe3fb11 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:26.968194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c6350379-a0aa-4982-ac9e-7476bbe3fb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:27.033998 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49d088-04da-41df-a934-9777b086053a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.065629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.069973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe452891-1143-4cd9-bcfd-53922a4e9352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.070753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe452891-1143-4cd9-bcfd-53922a4e9352 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.071168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe452891-1143-4cd9-bcfd-53922a4e9352 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.071576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe452891-1143-4cd9-bcfd-53922a4e9352 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.072336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe452891-1143-4cd9-bcfd-53922a4e9352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:27.072794 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe452891-1143-4cd9-bcfd-53922a4e9352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.075276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe452891-1143-4cd9-bcfd-53922a4e9352 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.078242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 229/464] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:27.080395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6350379-a0aa-4982-ac9e-7476bbe3fb11 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.112298 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 236/465] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:27.117764 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe452891-1143-4cd9-bcfd-53922a4e9352 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.194904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6350379-a0aa-4982-ac9e-7476bbe3fb11 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.236508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e940-ab80-4cf6-8c29-b3d548b212fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.237675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e940-ab80-4cf6-8c29-b3d548b212fb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.238125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e940-ab80-4cf6-8c29-b3d548b212fb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.238850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e940-ab80-4cf6-8c29-b3d548b212fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.246299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.253697 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4e940-ab80-4cf6-8c29-b3d548b212fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.255468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4e940-ab80-4cf6-8c29-b3d548b212fb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-b2b4e940-ab80-4cf6-8c29-b3d548b212fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:27.275730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:27.276676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:27.277537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:27.277537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:27.277537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:27.279985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 230/466] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:27.286209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e940-ab80-4cf6-8c29-b3d548b212fb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.309359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe452891-1143-4cd9-bcfd-53922a4e9352 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.345021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.347397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e8792-4ff3-4912-80b8-cc25eb8eff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.348721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e8792-4ff3-4912-80b8-cc25eb8eff39 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.349424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e8792-4ff3-4912-80b8-cc25eb8eff39 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.349895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e8792-4ff3-4912-80b8-cc25eb8eff39 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.350852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e8792-4ff3-4912-80b8-cc25eb8eff39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.352518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e8792-4ff3-4912-80b8-cc25eb8eff39 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.379138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 237/467] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:27 2022] POST /identity/v3/users => generated 246 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:48:27.380741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e8792-4ff3-4912-80b8-cc25eb8eff39 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.423633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e940-ab80-4cf6-8c29-b3d548b212fb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.463613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f440c-6d4e-4da8-93cc-d329057fff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.464319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f440c-6d4e-4da8-93cc-d329057fff0a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.464600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f440c-6d4e-4da8-93cc-d329057fff0a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.464869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f440c-6d4e-4da8-93cc-d329057fff0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.478354 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f440c-6d4e-4da8-93cc-d329057fff0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.480036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f440c-6d4e-4da8-93cc-d329057fff0a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.496270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e8792-4ff3-4912-80b8-cc25eb8eff39 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3d6f440c-6d4e-4da8-93cc-d329057fff0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:27.497420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:27.498284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:27.499096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:27.499096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:27.499096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:27.504207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 231/468] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:27.505273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f440c-6d4e-4da8-93cc-d329057fff0a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.509068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.540678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860c016-80d8-4f70-8e98-d0385e9ba26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.541589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860c016-80d8-4f70-8e98-d0385e9ba26e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.542048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860c016-80d8-4f70-8e98-d0385e9ba26e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.542318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860c016-80d8-4f70-8e98-d0385e9ba26e None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.542960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860c016-80d8-4f70-8e98-d0385e9ba26e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:27.543260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860c016-80d8-4f70-8e98-d0385e9ba26e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.544658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860c016-80d8-4f70-8e98-d0385e9ba26e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.557813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 238/469] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:27.558724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860c016-80d8-4f70-8e98-d0385e9ba26e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.632778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.640120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f440c-6d4e-4da8-93cc-d329057fff0a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.683464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20aca22-5251-46b1-bb4b-1c0be2c0d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.684092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20aca22-5251-46b1-bb4b-1c0be2c0d5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.684364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20aca22-5251-46b1-bb4b-1c0be2c0d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.684630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20aca22-5251-46b1-bb4b-1c0be2c0d5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.685869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860c016-80d8-4f70-8e98-d0385e9ba26e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.698120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20aca22-5251-46b1-bb4b-1c0be2c0d5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.699568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20aca22-5251-46b1-bb4b-1c0be2c0d5c2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d20aca22-5251-46b1-bb4b-1c0be2c0d5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:27.719839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:27.720783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:27.721662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:27.721662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:27.721662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:27.721662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 232/470] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:27.722968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20aca22-5251-46b1-bb4b-1c0be2c0d5c2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.728267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397609-f25d-495a-9dea-1c8a6ad815c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.729237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397609-f25d-495a-9dea-1c8a6ad815c1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.730524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397609-f25d-495a-9dea-1c8a6ad815c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.730806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397609-f25d-495a-9dea-1c8a6ad815c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.731459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44397609-f25d-495a-9dea-1c8a6ad815c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:27.731764 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44397609-f25d-495a-9dea-1c8a6ad815c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.733173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44397609-f25d-495a-9dea-1c8a6ad815c1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.749732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 239/471] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:27.750716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397609-f25d-495a-9dea-1c8a6ad815c1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.785025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.857592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20aca22-5251-46b1-bb4b-1c0be2c0d5c2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.863386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397609-f25d-495a-9dea-1c8a6ad815c1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.893531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01c848-0190-475b-8f4e-81218f79c6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.894128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01c848-0190-475b-8f4e-81218f79c6ba None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.894404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01c848-0190-475b-8f4e-81218f79c6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.894882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01c848-0190-475b-8f4e-81218f79c6ba None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.895302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01c848-0190-475b-8f4e-81218f79c6ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.896829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01c848-0190-475b-8f4e-81218f79c6ba None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.899656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b75a28-32c1-4074-9a98-8c5c345f5890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.900264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b75a28-32c1-4074-9a98-8c5c345f5890 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.900545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b75a28-32c1-4074-9a98-8c5c345f5890 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.900816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b75a28-32c1-4074-9a98-8c5c345f5890 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.901494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b75a28-32c1-4074-9a98-8c5c345f5890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:27.901869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b75a28-32c1-4074-9a98-8c5c345f5890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.903224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b75a28-32c1-4074-9a98-8c5c345f5890 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.921535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 240/472] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:27.921717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 233/473] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:27 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:27.921823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b75a28-32c1-4074-9a98-8c5c345f5890 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.921898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ae981-d1ba-480d-a0ac-db7bc4e7fe0a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.921898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ae981-d1ba-480d-a0ac-db7bc4e7fe0a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.921898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ae981-d1ba-480d-a0ac-db7bc4e7fe0a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.929630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 234/474] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:27.930594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ae981-d1ba-480d-a0ac-db7bc4e7fe0a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.939170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:28.059922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:28.072642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ae981-d1ba-480d-a0ac-db7bc4e7fe0a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:28.085500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b75a28-32c1-4074-9a98-8c5c345f5890 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:28.119370 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a83417-6996-456e-b3f9-e16cb6eed171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:28.119902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a83417-6996-456e-b3f9-e16cb6eed171 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.120180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a83417-6996-456e-b3f9-e16cb6eed171 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.120450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a83417-6996-456e-b3f9-e16cb6eed171 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.121761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a83417-6996-456e-b3f9-e16cb6eed171 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:28.122680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a83417-6996-456e-b3f9-e16cb6eed171 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:28.129932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2c299-6a75-49e6-8d55-fa26070147e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:28.130595 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2c299-6a75-49e6-8d55-fa26070147e4 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.130828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2c299-6a75-49e6-8d55-fa26070147e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.131097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2c299-6a75-49e6-8d55-fa26070147e4 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.131742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2c299-6a75-49e6-8d55-fa26070147e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:28.134333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2c299-6a75-49e6-8d55-fa26070147e4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:28.153156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 235/475] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:27 2022] POST /identity/v3/users => generated 249 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:28.159188 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed34618-43f6-4962-8a31-212188b3a42b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.159755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed34618-43f6-4962-8a31-212188b3a42b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.160225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed34618-43f6-4962-8a31-212188b3a42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.161062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 241/476] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:27 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:28.188256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-bed34618-43f6-4962-8a31-212188b3a42b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:28.191294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bed34618-43f6-4962-8a31-212188b3a42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:28.268667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:28.283562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 236/477] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:28.578302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2c299-6a75-49e6-8d55-fa26070147e4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:28.691973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2c299-6a75-49e6-8d55-fa26070147e4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:28.732248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:28.732873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.733177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.733454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.734128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:28.736671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:28.752373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.sql.core [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 00:48:28.752373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 00:48:28.752373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [parameters: {'id': '0111c772d2244558b05c7011a50b57dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 00:48:28.752373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81443) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result.read() Aug 30 00:48:28.768973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:48:28.769894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application yield resource Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise self.value Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application session.commit() Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 00:48:28.771088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.session.flush() Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application compat.raise_( Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 00:48:28.771933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:48:28.772799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result.read() Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:48:28.773625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [parameters: {'id': '0111c772d2244558b05c7011a50b57dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:28.774494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 00:48:28.779908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:28.783194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 242/478] 10.222.0.78 () {60 vars in 1131 bytes} [Tue Aug 30 00:48:28 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:28.783330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed34618-43f6-4962-8a31-212188b3a42b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:28.919516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed34618-43f6-4962-8a31-212188b3a42b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:28.959927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784c4ef-2b51-4b75-8575-6efdfa36b595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:28.960605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784c4ef-2b51-4b75-8575-6efdfa36b595 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.960884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784c4ef-2b51-4b75-8575-6efdfa36b595 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.961170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784c4ef-2b51-4b75-8575-6efdfa36b595 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.971723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784c4ef-2b51-4b75-8575-6efdfa36b595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-f784c4ef-2b51-4b75-8575-6efdfa36b595 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:28.976584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:28.978420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:28.979590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 237/479] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:28.983461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:29.005033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:29.120396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8758e8-cdb1-4bd6-8553-d58d4edba7c4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:29.162229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48811d3-a8c7-49a0-bd53-1f6a157a682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:29.163030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48811d3-a8c7-49a0-bd53-1f6a157a682e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:29.163189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48811d3-a8c7-49a0-bd53-1f6a157a682e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:29.163337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48811d3-a8c7-49a0-bd53-1f6a157a682e None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:29.164001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48811d3-a8c7-49a0-bd53-1f6a157a682e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:29.164308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48811d3-a8c7-49a0-bd53-1f6a157a682e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:29.165792 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48811d3-a8c7-49a0-bd53-1f6a157a682e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:29.180335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 243/480] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:29.617392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147d152-b1cc-4fe9-9b80-558d24774af5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:29.617392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147d152-b1cc-4fe9-9b80-558d24774af5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:29.617392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147d152-b1cc-4fe9-9b80-558d24774af5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:29.618633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 238/481] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:29.624887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b0889-89c8-4d82-a2c0-acd1d36d758c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:29.626631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b0889-89c8-4d82-a2c0-acd1d36d758c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:29.627122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b0889-89c8-4d82-a2c0-acd1d36d758c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:29.653969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ce9b0889-89c8-4d82-a2c0-acd1d36d758c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:29.655489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ce9b0889-89c8-4d82-a2c0-acd1d36d758c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:29.732268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:29.738151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 244/482] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:30.150503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147d152-b1cc-4fe9-9b80-558d24774af5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:30.230154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:30.304738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147d152-b1cc-4fe9-9b80-558d24774af5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:30.358389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131db83-b799-4303-ba9b-6e54d9b98ca2 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.359323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131db83-b799-4303-ba9b-6e54d9b98ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.359822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131db83-b799-4303-ba9b-6e54d9b98ca2 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.360752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676c2b1-bba5-4f7b-a263-649255c6ec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:30.361411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676c2b1-bba5-4f7b-a263-649255c6ec34 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.361687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676c2b1-bba5-4f7b-a263-649255c6ec34 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.361959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676c2b1-bba5-4f7b-a263-649255c6ec34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.362771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 245/483] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:30.368003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5731e37-6d19-489e-ba01-f880699e5855 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.368532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5731e37-6d19-489e-ba01-f880699e5855 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.368993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5731e37-6d19-489e-ba01-f880699e5855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.384163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676c2b1-bba5-4f7b-a263-649255c6ec34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:30.387811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b5731e37-6d19-489e-ba01-f880699e5855 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:30.389261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b5731e37-6d19-489e-ba01-f880699e5855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7676c2b1-bba5-4f7b-a263-649255c6ec34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:30.389367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:30.390206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:30.391566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 239/484] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:30.402127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676c2b1-bba5-4f7b-a263-649255c6ec34 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:30.507835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 246/485] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:30.575163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676c2b1-bba5-4f7b-a263-649255c6ec34 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:30.592133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:30.623726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fa7cd-4cc9-4062-9f17-55bd69de20a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:30.624340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fa7cd-4cc9-4062-9f17-55bd69de20a9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.624587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fa7cd-4cc9-4062-9f17-55bd69de20a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.624853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fa7cd-4cc9-4062-9f17-55bd69de20a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.626862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fa7cd-4cc9-4062-9f17-55bd69de20a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:30.627168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fa7cd-4cc9-4062-9f17-55bd69de20a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:30.628531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fa7cd-4cc9-4062-9f17-55bd69de20a9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:30.640631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 240/486] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:30.645870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5731e37-6d19-489e-ba01-f880699e5855 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:30.663970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ce5a2-dc21-4d2c-b4c6-f0b37cb16e1c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.664267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ce5a2-dc21-4d2c-b4c6-f0b37cb16e1c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.664450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ce5a2-dc21-4d2c-b4c6-f0b37cb16e1c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.666100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 241/487] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:30.674483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad117db2-7f2a-415d-bfe5-e54971629f40 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.674760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad117db2-7f2a-415d-bfe5-e54971629f40 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.675033 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad117db2-7f2a-415d-bfe5-e54971629f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.705513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ad117db2-7f2a-415d-bfe5-e54971629f40 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:30.706710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ad117db2-7f2a-415d-bfe5-e54971629f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:30.820066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5731e37-6d19-489e-ba01-f880699e5855 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:30.829594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 242/488] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:30.831623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad117db2-7f2a-415d-bfe5-e54971629f40 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:30.853642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:30.866444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce0da9-b946-47a5-be5a-8290520bc48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:30.867686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce0da9-b946-47a5-be5a-8290520bc48e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.868562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce0da9-b946-47a5-be5a-8290520bc48e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.869359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce0da9-b946-47a5-be5a-8290520bc48e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.883252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce0da9-b946-47a5-be5a-8290520bc48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:30.885437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce0da9-b946-47a5-be5a-8290520bc48e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-10ce0da9-b946-47a5-be5a-8290520bc48e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:30.909796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:30.910750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:30.911690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:30.911690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:30.911690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:30.914734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 247/489] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:30.925129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce0da9-b946-47a5-be5a-8290520bc48e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:30.990908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad117db2-7f2a-415d-bfe5-e54971629f40 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:30.992402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.031062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be48d0-be21-4908-a182-a8bdbd3cedcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.032651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be48d0-be21-4908-a182-a8bdbd3cedcb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.033101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be48d0-be21-4908-a182-a8bdbd3cedcb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.033741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be48d0-be21-4908-a182-a8bdbd3cedcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.049473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be48d0-be21-4908-a182-a8bdbd3cedcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a1be48d0-be21-4908-a182-a8bdbd3cedcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:31.056598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:31.057522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:31.058309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:31.058925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 243/490] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:31.059880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c728f2-fc27-4ff5-9c18-3247cb6e5e77 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.060156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c728f2-fc27-4ff5-9c18-3247cb6e5e77 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.060421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c728f2-fc27-4ff5-9c18-3247cb6e5e77 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.065865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 244/491] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:31.065865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c728f2-fc27-4ff5-9c18-3247cb6e5e77 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.069346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce0da9-b946-47a5-be5a-8290520bc48e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.114708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b4073-28b9-43cd-83ed-6f2081ca7260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.115905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b4073-28b9-43cd-83ed-6f2081ca7260 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.116689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b4073-28b9-43cd-83ed-6f2081ca7260 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.117264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b4073-28b9-43cd-83ed-6f2081ca7260 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.118141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b4073-28b9-43cd-83ed-6f2081ca7260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:31.118654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b4073-28b9-43cd-83ed-6f2081ca7260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.120969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b4073-28b9-43cd-83ed-6f2081ca7260 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.142462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 248/492] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:31.142988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa28766-54f8-4946-8142-d0cf7c95449f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.143093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa28766-54f8-4946-8142-d0cf7c95449f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.143327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa28766-54f8-4946-8142-d0cf7c95449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.161896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7fa28766-54f8-4946-8142-d0cf7c95449f None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:31.163052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7fa28766-54f8-4946-8142-d0cf7c95449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:31.175399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c728f2-fc27-4ff5-9c18-3247cb6e5e77 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.198784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.211316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2613b6d-cd5d-4829-83b5-c488bbea45f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.211917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2613b6d-cd5d-4829-83b5-c488bbea45f3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.212207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2613b6d-cd5d-4829-83b5-c488bbea45f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.212453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2613b6d-cd5d-4829-83b5-c488bbea45f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.213142 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2613b6d-cd5d-4829-83b5-c488bbea45f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:31.213443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2613b6d-cd5d-4829-83b5-c488bbea45f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.214792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2613b6d-cd5d-4829-83b5-c488bbea45f3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.223763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 245/493] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:31.229915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2613b6d-cd5d-4829-83b5-c488bbea45f3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.246022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 249/494] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:31.246022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa28766-54f8-4946-8142-d0cf7c95449f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.264585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.346143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2613b6d-cd5d-4829-83b5-c488bbea45f3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.374062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa28766-54f8-4946-8142-d0cf7c95449f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.383184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93902b95-fdf2-4afa-8159-dbebe44293d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.383949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93902b95-fdf2-4afa-8159-dbebe44293d3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.384324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93902b95-fdf2-4afa-8159-dbebe44293d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.384691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93902b95-fdf2-4afa-8159-dbebe44293d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.393657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93902b95-fdf2-4afa-8159-dbebe44293d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.395549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93902b95-fdf2-4afa-8159-dbebe44293d3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-93902b95-fdf2-4afa-8159-dbebe44293d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:31.409576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:31.411603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:31.413530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:31.417721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 246/495] 10.222.0.78 () {56 vars in 1093 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:31.418827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93902b95-fdf2-4afa-8159-dbebe44293d3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.441121 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9fc5f-0024-47af-9ad7-3b374bc317c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.441979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9fc5f-0024-47af-9ad7-3b374bc317c1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.442401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9fc5f-0024-47af-9ad7-3b374bc317c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.442821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9fc5f-0024-47af-9ad7-3b374bc317c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.457875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9fc5f-0024-47af-9ad7-3b374bc317c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-bfb9fc5f-0024-47af-9ad7-3b374bc317c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:31.463275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:31.464331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:31.465192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:31.466414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 250/496] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:31.468358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9fc5f-0024-47af-9ad7-3b374bc317c1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.510503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.518405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.540146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93902b95-fdf2-4afa-8159-dbebe44293d3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.575889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35347b-2591-4fd2-b4cf-4beda3f58367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.576535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35347b-2591-4fd2-b4cf-4beda3f58367 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.576811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35347b-2591-4fd2-b4cf-4beda3f58367 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.577093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35347b-2591-4fd2-b4cf-4beda3f58367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.588371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35347b-2591-4fd2-b4cf-4beda3f58367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.589849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35347b-2591-4fd2-b4cf-4beda3f58367 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.591660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9fc5f-0024-47af-9ad7-3b374bc317c1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-6e35347b-2591-4fd2-b4cf-4beda3f58367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:31.604871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:31.605921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:31.606841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:31.606841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:31.606841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:31.606841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 247/497] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:31.606841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35347b-2591-4fd2-b4cf-4beda3f58367 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.636091 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b153ebe-7cdc-482c-a02a-b26b7d4b5f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.636787 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b153ebe-7cdc-482c-a02a-b26b7d4b5f26 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.637163 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b153ebe-7cdc-482c-a02a-b26b7d4b5f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.637435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b153ebe-7cdc-482c-a02a-b26b7d4b5f26 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.638218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b153ebe-7cdc-482c-a02a-b26b7d4b5f26 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:31.638518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b153ebe-7cdc-482c-a02a-b26b7d4b5f26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.639979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b153ebe-7cdc-482c-a02a-b26b7d4b5f26 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.677201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 251/498] 10.222.0.78 () {56 vars in 1133 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:31.678044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b153ebe-7cdc-482c-a02a-b26b7d4b5f26 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.725216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35347b-2591-4fd2-b4cf-4beda3f58367 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.760636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a12948-e50d-406a-b834-accc33e60bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.761271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a12948-e50d-406a-b834-accc33e60bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.761555 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a12948-e50d-406a-b834-accc33e60bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.761828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a12948-e50d-406a-b834-accc33e60bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.762614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a12948-e50d-406a-b834-accc33e60bb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:31.762919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a12948-e50d-406a-b834-accc33e60bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.764412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a12948-e50d-406a-b834-accc33e60bb7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.767457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.775672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.780968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 248/499] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:31.780968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a12948-e50d-406a-b834-accc33e60bb7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.794615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b153ebe-7cdc-482c-a02a-b26b7d4b5f26 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.830441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b83fc0-2f80-4998-8681-fdf089f01e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.831063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b83fc0-2f80-4998-8681-fdf089f01e80 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.831337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b83fc0-2f80-4998-8681-fdf089f01e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.831602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b83fc0-2f80-4998-8681-fdf089f01e80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.841484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b83fc0-2f80-4998-8681-fdf089f01e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-64b83fc0-2f80-4998-8681-fdf089f01e80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:31.846895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:31.847930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:31.848831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 252/500] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:31.849758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b83fc0-2f80-4998-8681-fdf089f01e80 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.886145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a12948-e50d-406a-b834-accc33e60bb7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.923679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c00afc-ce06-4684-a96f-0cb973be0d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.924404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c00afc-ce06-4684-a96f-0cb973be0d1d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.924690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c00afc-ce06-4684-a96f-0cb973be0d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.924962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c00afc-ce06-4684-a96f-0cb973be0d1d None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.925638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c00afc-ce06-4684-a96f-0cb973be0d1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:31.925940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c00afc-ce06-4684-a96f-0cb973be0d1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.927292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c00afc-ce06-4684-a96f-0cb973be0d1d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.939121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 249/501] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:31.939919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c00afc-ce06-4684-a96f-0cb973be0d1d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.955860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b83fc0-2f80-4998-8681-fdf089f01e80 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.990833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7693d-aa4e-4a2e-a2df-1d7e804a5d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.991359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7693d-aa4e-4a2e-a2df-1d7e804a5d29 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.991638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7693d-aa4e-4a2e-a2df-1d7e804a5d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.991907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7693d-aa4e-4a2e-a2df-1d7e804a5d29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.998937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.010484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.040736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c00afc-ce06-4684-a96f-0cb973be0d1d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.060364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7693d-aa4e-4a2e-a2df-1d7e804a5d29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.061836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7693d-aa4e-4a2e-a2df-1d7e804a5d29 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.075305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65350f5c-1851-45db-a570-770fa8c1a1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.075925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65350f5c-1851-45db-a570-770fa8c1a1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.076201 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65350f5c-1851-45db-a570-770fa8c1a1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.076472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65350f5c-1851-45db-a570-770fa8c1a1d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.087462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65350f5c-1851-45db-a570-770fa8c1a1d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.088873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65350f5c-1851-45db-a570-770fa8c1a1d6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-65350f5c-1851-45db-a570-770fa8c1a1d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:32.104913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:32.106025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:32.107050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:32.107050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:32.107050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:32.107050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 250/502] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:32.107379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee200130-2cf8-4384-a98b-7be4c84b29d3 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.107466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee200130-2cf8-4384-a98b-7be4c84b29d3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.107680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee200130-2cf8-4384-a98b-7be4c84b29d3 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.109172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 251/503] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:32.110575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee200130-2cf8-4384-a98b-7be4c84b29d3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.163820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 253/504] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:32.164686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7693d-aa4e-4a2e-a2df-1d7e804a5d29 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.213157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.214053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee200130-2cf8-4384-a98b-7be4c84b29d3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.224684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.248945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe7253-d863-44b1-b907-429e128b031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.249497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe7253-d863-44b1-b907-429e128b031c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.249771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe7253-d863-44b1-b907-429e128b031c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.250038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe7253-d863-44b1-b907-429e128b031c None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.251323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe7253-d863-44b1-b907-429e128b031c None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:32.251619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe7253-d863-44b1-b907-429e128b031c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.252994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe7253-d863-44b1-b907-429e128b031c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.261532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 252/505] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:32.262322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe7253-d863-44b1-b907-429e128b031c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.272952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7693d-aa4e-4a2e-a2df-1d7e804a5d29 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.306668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38824b3-1253-4287-906b-16b4dc5e5cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.307286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38824b3-1253-4287-906b-16b4dc5e5cff None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.307561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38824b3-1253-4287-906b-16b4dc5e5cff None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.307832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38824b3-1253-4287-906b-16b4dc5e5cff None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.315458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38824b3-1253-4287-906b-16b4dc5e5cff None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.316885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38824b3-1253-4287-906b-16b4dc5e5cff None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c38824b3-1253-4287-906b-16b4dc5e5cff None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:32.329776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:32.330841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:32.331913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 254/506] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:32.332585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b512c-712c-4264-b368-65d640673263 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.332585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b512c-712c-4264-b368-65d640673263 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.332585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b512c-712c-4264-b368-65d640673263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.351667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-526b512c-712c-4264-b368-65d640673263 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:32.352844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-526b512c-712c-4264-b368-65d640673263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:32.361973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe7253-d863-44b1-b907-429e128b031c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.395655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffebd5-4e3a-4791-aeba-a99552a97222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.396253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffebd5-4e3a-4791-aeba-a99552a97222 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.396532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffebd5-4e3a-4791-aeba-a99552a97222 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.396803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffebd5-4e3a-4791-aeba-a99552a97222 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.397477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffebd5-4e3a-4791-aeba-a99552a97222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:32.397780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffebd5-4e3a-4791-aeba-a99552a97222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.399137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffebd5-4e3a-4791-aeba-a99552a97222 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.411967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 253/507] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:32.412911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffebd5-4e3a-4791-aeba-a99552a97222 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.425032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 255/508] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:32.425827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b512c-712c-4264-b368-65d640673263 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.444928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.476396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.524268 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffebd5-4e3a-4791-aeba-a99552a97222 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.537967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b512c-712c-4264-b368-65d640673263 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.564972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3067d8-b0c1-4f04-81ee-5d9c5bc21f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.565623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3067d8-b0c1-4f04-81ee-5d9c5bc21f44 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.565897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3067d8-b0c1-4f04-81ee-5d9c5bc21f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.566167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3067d8-b0c1-4f04-81ee-5d9c5bc21f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.572120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcaefe-4b01-4244-9a55-8d6d33f24bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.572738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcaefe-4b01-4244-9a55-8d6d33f24bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.573013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcaefe-4b01-4244-9a55-8d6d33f24bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.573314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcaefe-4b01-4244-9a55-8d6d33f24bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.577534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3067d8-b0c1-4f04-81ee-5d9c5bc21f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.579284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3067d8-b0c1-4f04-81ee-5d9c5bc21f44 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.583415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcaefe-4b01-4244-9a55-8d6d33f24bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.584860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcaefe-4b01-4244-9a55-8d6d33f24bf3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-0b3067d8-b0c1-4f04-81ee-5d9c5bc21f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:32.594834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:32.595891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:32.596940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:32.596940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:32.596940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:32.596940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 254/509] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:32.601392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3067d8-b0c1-4f04-81ee-5d9c5bc21f44 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-24bcaefe-4b01-4244-9a55-8d6d33f24bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:32.602664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:32.603696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:32.610565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:32.610565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:32.610565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:32.610565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 256/510] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:32.610565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcaefe-4b01-4244-9a55-8d6d33f24bf3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.698793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.703719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.713134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcaefe-4b01-4244-9a55-8d6d33f24bf3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.715918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3067d8-b0c1-4f04-81ee-5d9c5bc21f44 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.747035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07252b24-00c2-45f5-a347-e381c3e324e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.747656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07252b24-00c2-45f5-a347-e381c3e324e3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.747930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07252b24-00c2-45f5-a347-e381c3e324e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.748196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07252b24-00c2-45f5-a347-e381c3e324e3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.750038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a133abe-7146-40ec-b7f9-e8576e224f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.750814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a133abe-7146-40ec-b7f9-e8576e224f7a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.751091 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a133abe-7146-40ec-b7f9-e8576e224f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.751362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a133abe-7146-40ec-b7f9-e8576e224f7a None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.752080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a133abe-7146-40ec-b7f9-e8576e224f7a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:32.752379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a133abe-7146-40ec-b7f9-e8576e224f7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.753817 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a133abe-7146-40ec-b7f9-e8576e224f7a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.755622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07252b24-00c2-45f5-a347-e381c3e324e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.757107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07252b24-00c2-45f5-a347-e381c3e324e3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-07252b24-00c2-45f5-a347-e381c3e324e3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:32.768780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:32.769846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 257/511] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:32.770917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07252b24-00c2-45f5-a347-e381c3e324e3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.793797 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 255/512] 10.222.0.78 () {56 vars in 1137 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:32.794772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a133abe-7146-40ec-b7f9-e8576e224f7a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.873176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07252b24-00c2-45f5-a347-e381c3e324e3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.895054 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a133abe-7146-40ec-b7f9-e8576e224f7a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.907989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e5b88-5f58-41b5-b271-b1bc7032e93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.908585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e5b88-5f58-41b5-b271-b1bc7032e93a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.908865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e5b88-5f58-41b5-b271-b1bc7032e93a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.909160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e5b88-5f58-41b5-b271-b1bc7032e93a None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.909794 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e5b88-5f58-41b5-b271-b1bc7032e93a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:32.910095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e5b88-5f58-41b5-b271-b1bc7032e93a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.911452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e5b88-5f58-41b5-b271-b1bc7032e93a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.923123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 258/513] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:32.923976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e5b88-5f58-41b5-b271-b1bc7032e93a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.930693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f77ccd-4ab7-4313-ba3c-5d58b3dd2c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.931617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f77ccd-4ab7-4313-ba3c-5d58b3dd2c53 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.932051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f77ccd-4ab7-4313-ba3c-5d58b3dd2c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.932490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f77ccd-4ab7-4313-ba3c-5d58b3dd2c53 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.933573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f77ccd-4ab7-4313-ba3c-5d58b3dd2c53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:32.934065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f77ccd-4ab7-4313-ba3c-5d58b3dd2c53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.936409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f77ccd-4ab7-4313-ba3c-5d58b3dd2c53 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.941538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.950221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 256/514] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:32.955014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f77ccd-4ab7-4313-ba3c-5d58b3dd2c53 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.958726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.030757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e5b88-5f58-41b5-b271-b1bc7032e93a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.054253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f77ccd-4ab7-4313-ba3c-5d58b3dd2c53 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.064762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa50ab-be2c-4ac1-abc0-f32f0910b628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.065410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa50ab-be2c-4ac1-abc0-f32f0910b628 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.065689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa50ab-be2c-4ac1-abc0-f32f0910b628 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.065958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa50ab-be2c-4ac1-abc0-f32f0910b628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.075965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa50ab-be2c-4ac1-abc0-f32f0910b628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.077437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa50ab-be2c-4ac1-abc0-f32f0910b628 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.087788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6e74f-202d-4c00-80fb-7e3cf63a3be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.088374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6e74f-202d-4c00-80fb-7e3cf63a3be5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.088649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6e74f-202d-4c00-80fb-7e3cf63a3be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.088915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6e74f-202d-4c00-80fb-7e3cf63a3be5 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.089665 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6e74f-202d-4c00-80fb-7e3cf63a3be5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:33.089967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6e74f-202d-4c00-80fb-7e3cf63a3be5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.091358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6e74f-202d-4c00-80fb-7e3cf63a3be5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-f0aa50ab-be2c-4ac1-abc0-f32f0910b628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:33.092352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:33.093367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:33.094983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:33.094983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:33.094983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:33.094983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 259/515] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:33.094983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa50ab-be2c-4ac1-abc0-f32f0910b628 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.111695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 257/516] 10.222.0.78 () {56 vars in 1137 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.112517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6e74f-202d-4c00-80fb-7e3cf63a3be5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.155039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.173570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.194250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa50ab-be2c-4ac1-abc0-f32f0910b628 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.214653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6e74f-202d-4c00-80fb-7e3cf63a3be5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.227629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639473aa-d407-4de5-9eb4-686dbe4e63d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.228226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639473aa-d407-4de5-9eb4-686dbe4e63d7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.228507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639473aa-d407-4de5-9eb4-686dbe4e63d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.228777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639473aa-d407-4de5-9eb4-686dbe4e63d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.251209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838b2ae-9e4e-45a7-8a01-170e162e80d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.251835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838b2ae-9e4e-45a7-8a01-170e162e80d1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.252113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838b2ae-9e4e-45a7-8a01-170e162e80d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.252381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838b2ae-9e4e-45a7-8a01-170e162e80d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.259723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838b2ae-9e4e-45a7-8a01-170e162e80d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.261271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838b2ae-9e4e-45a7-8a01-170e162e80d1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d838b2ae-9e4e-45a7-8a01-170e162e80d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:33.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:33.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:33.275196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:33.275196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:33.275196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:33.275196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:33.275196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:33.275196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:33.275196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:33.275196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:33.275196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 258/517] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:33.275643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838b2ae-9e4e-45a7-8a01-170e162e80d1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.300239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639473aa-d407-4de5-9eb4-686dbe4e63d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.301744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639473aa-d407-4de5-9eb4-686dbe4e63d7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.362192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.376010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838b2ae-9e4e-45a7-8a01-170e162e80d1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.395773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.404287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 260/518] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:33.405166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639473aa-d407-4de5-9eb4-686dbe4e63d7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.413474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410e0be-6869-4871-b929-f79c88f1f481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.414101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410e0be-6869-4871-b929-f79c88f1f481 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.414379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410e0be-6869-4871-b929-f79c88f1f481 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.414654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410e0be-6869-4871-b929-f79c88f1f481 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.422026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410e0be-6869-4871-b929-f79c88f1f481 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.423907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410e0be-6869-4871-b929-f79c88f1f481 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a410e0be-6869-4871-b929-f79c88f1f481 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:33.434818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:33.435895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 259/519] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:33.437362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410e0be-6869-4871-b929-f79c88f1f481 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.509662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639473aa-d407-4de5-9eb4-686dbe4e63d7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.539412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410e0be-6869-4871-b929-f79c88f1f481 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.543441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a45ef-1942-49f5-b222-093433b1202e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.544043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a45ef-1942-49f5-b222-093433b1202e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.544320 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a45ef-1942-49f5-b222-093433b1202e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.544586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a45ef-1942-49f5-b222-093433b1202e None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.545250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a45ef-1942-49f5-b222-093433b1202e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:33.545550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a45ef-1942-49f5-b222-093433b1202e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.547436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a45ef-1942-49f5-b222-093433b1202e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.558490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 261/520] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.559322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a45ef-1942-49f5-b222-093433b1202e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.574500 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17de934-1e59-48f5-860a-d897031e75d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.575096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17de934-1e59-48f5-860a-d897031e75d7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.575374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17de934-1e59-48f5-860a-d897031e75d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.575642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17de934-1e59-48f5-860a-d897031e75d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.585277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.625162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.646926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17de934-1e59-48f5-860a-d897031e75d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.648387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17de934-1e59-48f5-860a-d897031e75d7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.658892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a45ef-1942-49f5-b222-093433b1202e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.692900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448af57a-11d8-42b3-b8fb-bfb0769f18e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.693663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448af57a-11d8-42b3-b8fb-bfb0769f18e2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.693942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448af57a-11d8-42b3-b8fb-bfb0769f18e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.694215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448af57a-11d8-42b3-b8fb-bfb0769f18e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.695154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448af57a-11d8-42b3-b8fb-bfb0769f18e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:33.695457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448af57a-11d8-42b3-b8fb-bfb0769f18e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.696902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448af57a-11d8-42b3-b8fb-bfb0769f18e2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.714134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 262/521] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.714940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448af57a-11d8-42b3-b8fb-bfb0769f18e2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.748410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 260/522] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:33.749282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17de934-1e59-48f5-860a-d897031e75d7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.800706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.820357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448af57a-11d8-42b3-b8fb-bfb0769f18e2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.832800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.852236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17de934-1e59-48f5-860a-d897031e75d7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.854362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a40ad-77f5-47f9-8018-82953572e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.854989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a40ad-77f5-47f9-8018-82953572e313 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.855271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a40ad-77f5-47f9-8018-82953572e313 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.855546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a40ad-77f5-47f9-8018-82953572e313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.866030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a40ad-77f5-47f9-8018-82953572e313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.867483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a40ad-77f5-47f9-8018-82953572e313 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-160a40ad-77f5-47f9-8018-82953572e313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:33.882709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:33.883770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:33.884752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:33.884752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:33.884752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:33.884752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 263/523] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:33.884752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a40ad-77f5-47f9-8018-82953572e313 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.888347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad175256-e188-4dd5-8ec3-6acb52dc0906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.888997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad175256-e188-4dd5-8ec3-6acb52dc0906 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.889296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad175256-e188-4dd5-8ec3-6acb52dc0906 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.889561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad175256-e188-4dd5-8ec3-6acb52dc0906 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.890281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad175256-e188-4dd5-8ec3-6acb52dc0906 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:33.890580 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad175256-e188-4dd5-8ec3-6acb52dc0906 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.892146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad175256-e188-4dd5-8ec3-6acb52dc0906 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.909776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 261/524] 10.222.0.78 () {56 vars in 1139 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.910604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad175256-e188-4dd5-8ec3-6acb52dc0906 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.983541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a40ad-77f5-47f9-8018-82953572e313 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.007771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad175256-e188-4dd5-8ec3-6acb52dc0906 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.018996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d94c-0062-412d-a9c7-4d035a223e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.019594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d94c-0062-412d-a9c7-4d035a223e6c None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.019872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d94c-0062-412d-a9c7-4d035a223e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.020141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d94c-0062-412d-a9c7-4d035a223e6c None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.020777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8d94c-0062-412d-a9c7-4d035a223e6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.022242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8d94c-0062-412d-a9c7-4d035a223e6c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.030114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.042526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85719b69-ef77-4aa8-a8e0-bdcdc13c5767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.043125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85719b69-ef77-4aa8-a8e0-bdcdc13c5767 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.043404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85719b69-ef77-4aa8-a8e0-bdcdc13c5767 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.043673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85719b69-ef77-4aa8-a8e0-bdcdc13c5767 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.044377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 264/525] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:33 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:34.044470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85719b69-ef77-4aa8-a8e0-bdcdc13c5767 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=f9b88c495fcb47a289a6a45276c891e0) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:34.044694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85719b69-ef77-4aa8-a8e0-bdcdc13c5767 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.045308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d94c-0062-412d-a9c7-4d035a223e6c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.046134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85719b69-ef77-4aa8-a8e0-bdcdc13c5767 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.052721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 262/526] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/role_assignments?user.id=f9b88c495fcb47a289a6a45276c891e0&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.053622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85719b69-ef77-4aa8-a8e0-bdcdc13c5767 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.056784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.205245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85719b69-ef77-4aa8-a8e0-bdcdc13c5767 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.216801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d94c-0062-412d-a9c7-4d035a223e6c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.256910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4f763-978a-454c-857c-ba375c237666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.257137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4f763-978a-454c-857c-ba375c237666 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.257386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4f763-978a-454c-857c-ba375c237666 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.257569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4f763-978a-454c-857c-ba375c237666 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.259229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4f763-978a-454c-857c-ba375c237666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:34.259229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4f763-978a-454c-857c-ba375c237666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.259886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4f763-978a-454c-857c-ba375c237666 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.267963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d4401-463e-41f6-9347-3e5c18fecb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.268753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d4401-463e-41f6-9347-3e5c18fecb2f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.269044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d4401-463e-41f6-9347-3e5c18fecb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.269362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d4401-463e-41f6-9347-3e5c18fecb2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.276401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 263/527] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.277517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4f763-978a-454c-857c-ba375c237666 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.282988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d4401-463e-41f6-9347-3e5c18fecb2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.284457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d4401-463e-41f6-9347-3e5c18fecb2f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-269d4401-463e-41f6-9347-3e5c18fecb2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:34.301353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:34.302304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:34.303160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:34.303160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:34.303160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:34.303160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 265/528] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:34.304291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d4401-463e-41f6-9347-3e5c18fecb2f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.352112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.355456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.395444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4f763-978a-454c-857c-ba375c237666 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.418542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d4401-463e-41f6-9347-3e5c18fecb2f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.431910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062e9b1-765a-460c-8d19-4b2cb78eed4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.432509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062e9b1-765a-460c-8d19-4b2cb78eed4f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.433061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062e9b1-765a-460c-8d19-4b2cb78eed4f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.433365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062e9b1-765a-460c-8d19-4b2cb78eed4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.456824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655f13-1ed7-486d-97e8-280d1862e415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.457737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655f13-1ed7-486d-97e8-280d1862e415 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.458223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655f13-1ed7-486d-97e8-280d1862e415 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.458680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655f13-1ed7-486d-97e8-280d1862e415 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.466028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655f13-1ed7-486d-97e8-280d1862e415 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.467635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655f13-1ed7-486d-97e8-280d1862e415 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-7c655f13-1ed7-486d-97e8-280d1862e415 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:34.480352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:34.481442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:34.482385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:34.482385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:34.482385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:34.482385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:34.482385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:34.482385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:34.482385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:34.482385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:34.484308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 266/529] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:34.486246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655f13-1ed7-486d-97e8-280d1862e415 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.511488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062e9b1-765a-460c-8d19-4b2cb78eed4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.513174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062e9b1-765a-460c-8d19-4b2cb78eed4f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.592845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655f13-1ed7-486d-97e8-280d1862e415 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.595129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.601827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.627371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 264/530] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:34.627371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062e9b1-765a-460c-8d19-4b2cb78eed4f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.635610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1821da3-d926-4ce8-949d-767e88b0661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.636452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1821da3-d926-4ce8-949d-767e88b0661e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.636929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1821da3-d926-4ce8-949d-767e88b0661e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.637360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1821da3-d926-4ce8-949d-767e88b0661e None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.638130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1821da3-d926-4ce8-949d-767e88b0661e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:34.638569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1821da3-d926-4ce8-949d-767e88b0661e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.640066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1821da3-d926-4ce8-949d-767e88b0661e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.654372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 267/531] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.655649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1821da3-d926-4ce8-949d-767e88b0661e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.732802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062e9b1-765a-460c-8d19-4b2cb78eed4f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.771516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8ac6-91d6-44af-a7d3-fdec782623b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.772129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8ac6-91d6-44af-a7d3-fdec782623b0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.772378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8ac6-91d6-44af-a7d3-fdec782623b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.772646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8ac6-91d6-44af-a7d3-fdec782623b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.773371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe8ac6-91d6-44af-a7d3-fdec782623b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:34.773673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe8ac6-91d6-44af-a7d3-fdec782623b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.775108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe8ac6-91d6-44af-a7d3-fdec782623b0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.792985 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1821da3-d926-4ce8-949d-767e88b0661e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.793474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 265/532] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.794291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8ac6-91d6-44af-a7d3-fdec782623b0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.825849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.841314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56eaaf-99af-4d05-adbb-4c384c083b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.842187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56eaaf-99af-4d05-adbb-4c384c083b2c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.842603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56eaaf-99af-4d05-adbb-4c384c083b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.843008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56eaaf-99af-4d05-adbb-4c384c083b2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.858667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56eaaf-99af-4d05-adbb-4c384c083b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.861720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56eaaf-99af-4d05-adbb-4c384c083b2c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3b56eaaf-99af-4d05-adbb-4c384c083b2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:34.881241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:34.882283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:34.883194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:34.883194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:34.883194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:34.889988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 268/533] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:34.889988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56eaaf-99af-4d05-adbb-4c384c083b2c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.913264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.914541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8ac6-91d6-44af-a7d3-fdec782623b0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.955194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc75975-0513-495e-8510-e75c51dd00b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.955815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc75975-0513-495e-8510-e75c51dd00b8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.956136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc75975-0513-495e-8510-e75c51dd00b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.956404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc75975-0513-495e-8510-e75c51dd00b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.964064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc75975-0513-495e-8510-e75c51dd00b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.965524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc75975-0513-495e-8510-e75c51dd00b8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-efc75975-0513-495e-8510-e75c51dd00b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:34.978207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:34.979053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:34.979889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:34.979889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:34.979889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:34.979889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:34.979889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:34.979889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:34.979889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:34.979889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:34.979889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 266/534] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:34.980342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc75975-0513-495e-8510-e75c51dd00b8 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.031179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56eaaf-99af-4d05-adbb-4c384c083b2c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.076300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dfd2c-0043-46d0-bf6c-f12bacf32040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.077294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dfd2c-0043-46d0-bf6c-f12bacf32040 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.077860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dfd2c-0043-46d0-bf6c-f12bacf32040 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.079088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dfd2c-0043-46d0-bf6c-f12bacf32040 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.080054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dfd2c-0043-46d0-bf6c-f12bacf32040 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=97e64ee3c8ee48c2ad8f351fb2a78bc8) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:35.080592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dfd2c-0043-46d0-bf6c-f12bacf32040 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.082394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dfd2c-0043-46d0-bf6c-f12bacf32040 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.088042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.089589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 269/535] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/role_assignments?user.id=97e64ee3c8ee48c2ad8f351fb2a78bc8&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.090605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dfd2c-0043-46d0-bf6c-f12bacf32040 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.147367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc75975-0513-495e-8510-e75c51dd00b8 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.182897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.211308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dfd2c-0043-46d0-bf6c-f12bacf32040 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.253713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24700651-3e5f-48a8-a88b-3acc2ae136e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.254698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24700651-3e5f-48a8-a88b-3acc2ae136e5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.255215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24700651-3e5f-48a8-a88b-3acc2ae136e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.255706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24700651-3e5f-48a8-a88b-3acc2ae136e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.256623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24700651-3e5f-48a8-a88b-3acc2ae136e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:35.257180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24700651-3e5f-48a8-a88b-3acc2ae136e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.257829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ecde3-13ce-4fb6-8bd6-5e397ea2f756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.258449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ecde3-13ce-4fb6-8bd6-5e397ea2f756 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.258725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ecde3-13ce-4fb6-8bd6-5e397ea2f756 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.258993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ecde3-13ce-4fb6-8bd6-5e397ea2f756 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.259648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ecde3-13ce-4fb6-8bd6-5e397ea2f756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:35.259946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ecde3-13ce-4fb6-8bd6-5e397ea2f756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.260297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24700651-3e5f-48a8-a88b-3acc2ae136e5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.261343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ecde3-13ce-4fb6-8bd6-5e397ea2f756 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.275891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 270/536] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.286331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24700651-3e5f-48a8-a88b-3acc2ae136e5 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.295358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 267/537] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.306169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ecde3-13ce-4fb6-8bd6-5e397ea2f756 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.407577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24700651-3e5f-48a8-a88b-3acc2ae136e5 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.444878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a59c-cb2f-4bfd-90d7-0d532169e20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.445951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a59c-cb2f-4bfd-90d7-0d532169e20b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.446438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a59c-cb2f-4bfd-90d7-0d532169e20b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.446888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a59c-cb2f-4bfd-90d7-0d532169e20b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.447888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a59c-cb2f-4bfd-90d7-0d532169e20b None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=77a47e77cc1942618a3584862042c5f7) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:35.448377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a59c-cb2f-4bfd-90d7-0d532169e20b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.450113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a59c-cb2f-4bfd-90d7-0d532169e20b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.457030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 271/538] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/role_assignments?user.id=77a47e77cc1942618a3584862042c5f7&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.482683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ecde3-13ce-4fb6-8bd6-5e397ea2f756 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.526246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3504c3-5e57-4402-bc59-90e0ca5b8467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.527210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3504c3-5e57-4402-bc59-90e0ca5b8467 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.527685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3504c3-5e57-4402-bc59-90e0ca5b8467 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.528149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3504c3-5e57-4402-bc59-90e0ca5b8467 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.539372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3504c3-5e57-4402-bc59-90e0ca5b8467 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.541180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3504c3-5e57-4402-bc59-90e0ca5b8467 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-cf3504c3-5e57-4402-bc59-90e0ca5b8467 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:35.558356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:35.559284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:35.560102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:35.560102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:35.560102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:35.560102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:35.560102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:35.560102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:35.560102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:35.560102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:35.562246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 268/539] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:35.568300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a59c-cb2f-4bfd-90d7-0d532169e20b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.581457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.723745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a59c-cb2f-4bfd-90d7-0d532169e20b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.761480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720c43-56eb-45ff-b3a0-245b47033b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.762093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720c43-56eb-45ff-b3a0-245b47033b64 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.762373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720c43-56eb-45ff-b3a0-245b47033b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.762642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720c43-56eb-45ff-b3a0-245b47033b64 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.763333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41720c43-56eb-45ff-b3a0-245b47033b64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:35.763638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41720c43-56eb-45ff-b3a0-245b47033b64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.765097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41720c43-56eb-45ff-b3a0-245b47033b64 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.777969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 272/540] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.785442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3504c3-5e57-4402-bc59-90e0ca5b8467 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.788463 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.911119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3504c3-5e57-4402-bc59-90e0ca5b8467 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.945949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a717f-eac2-41a8-8113-b2d774207367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.946566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a717f-eac2-41a8-8113-b2d774207367 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.946869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a717f-eac2-41a8-8113-b2d774207367 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.947143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a717f-eac2-41a8-8113-b2d774207367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.947863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a717f-eac2-41a8-8113-b2d774207367 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=1a5d841380c243209f55314b545ff56f) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:35.948162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a717f-eac2-41a8-8113-b2d774207367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.949675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a717f-eac2-41a8-8113-b2d774207367 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.955980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 269/541] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/role_assignments?user.id=1a5d841380c243209f55314b545ff56f&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:36.301726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34ba44-0c56-484c-88a4-471d139afe31 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.302053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34ba44-0c56-484c-88a4-471d139afe31 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.302282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34ba44-0c56-484c-88a4-471d139afe31 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.303919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 273/542] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:36.314615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea77ab9-a9e6-4160-8cfb-377ee8c12d57 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.315170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea77ab9-a9e6-4160-8cfb-377ee8c12d57 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.315657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea77ab9-a9e6-4160-8cfb-377ee8c12d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.342051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6ea77ab9-a9e6-4160-8cfb-377ee8c12d57 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:36.343573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6ea77ab9-a9e6-4160-8cfb-377ee8c12d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:36.413413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:36.423266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 270/543] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:36.428151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fdef5-f09b-4f61-820d-f9257e6275d9 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.428662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fdef5-f09b-4f61-820d-f9257e6275d9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.429159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fdef5-f09b-4f61-820d-f9257e6275d9 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.430902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 274/544] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:36.436171 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98233c54-d4cd-470a-a49b-a5e2ec35d91c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.436708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98233c54-d4cd-470a-a49b-a5e2ec35d91c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.437232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98233c54-d4cd-470a-a49b-a5e2ec35d91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.457271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-98233c54-d4cd-470a-a49b-a5e2ec35d91c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:36.458849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-98233c54-d4cd-470a-a49b-a5e2ec35d91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:36.552816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 271/545] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:36.874454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fdef5-f09b-4f61-820d-f9257e6275d9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:36.942110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:36.958486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98233c54-d4cd-470a-a49b-a5e2ec35d91c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:36.990600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fdef5-f09b-4f61-820d-f9257e6275d9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:37.047707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f4a3c-af49-4b5d-b6dc-ea117029de89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:37.048333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f4a3c-af49-4b5d-b6dc-ea117029de89 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.048608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f4a3c-af49-4b5d-b6dc-ea117029de89 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.048878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f4a3c-af49-4b5d-b6dc-ea117029de89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.073047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f4a3c-af49-4b5d-b6dc-ea117029de89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4a9f4a3c-af49-4b5d-b6dc-ea117029de89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:37.077850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:37.079317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:37.080098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:37.081570 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 275/546] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:37.087858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f4a3c-af49-4b5d-b6dc-ea117029de89 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:37.147844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98233c54-d4cd-470a-a49b-a5e2ec35d91c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:37.197843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:37.217421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10291c-1d61-4a53-830b-e27a948158fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:37.218050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10291c-1d61-4a53-830b-e27a948158fc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.218328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10291c-1d61-4a53-830b-e27a948158fc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.218596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10291c-1d61-4a53-830b-e27a948158fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.239611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10291c-1d61-4a53-830b-e27a948158fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-dc10291c-1d61-4a53-830b-e27a948158fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:37.244429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:37.245315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:37.246110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 272/547] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:37.252224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10291c-1d61-4a53-830b-e27a948158fc None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:37.293033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f4a3c-af49-4b5d-b6dc-ea117029de89 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:37.347013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:37.361501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10291c-1d61-4a53-830b-e27a948158fc None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:37.377451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8988276-4d7c-46f6-96a9-afd280f1e214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:37.378063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8988276-4d7c-46f6-96a9-afd280f1e214 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.378338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8988276-4d7c-46f6-96a9-afd280f1e214 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.378607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8988276-4d7c-46f6-96a9-afd280f1e214 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.379371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8988276-4d7c-46f6-96a9-afd280f1e214 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:37.379754 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8988276-4d7c-46f6-96a9-afd280f1e214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:37.381173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8988276-4d7c-46f6-96a9-afd280f1e214 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:37.397064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fabef-6ded-4119-9ae1-a9471c11de8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:37.397729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fabef-6ded-4119-9ae1-a9471c11de8b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.398073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fabef-6ded-4119-9ae1-a9471c11de8b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.398282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fabef-6ded-4119-9ae1-a9471c11de8b None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.398936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fabef-6ded-4119-9ae1-a9471c11de8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:37.399237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fabef-6ded-4119-9ae1-a9471c11de8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:37.400443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 276/548] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:37.400770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fabef-6ded-4119-9ae1-a9471c11de8b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:37.406120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8988276-4d7c-46f6-96a9-afd280f1e214 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:37.414799 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 273/549] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:37.416057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fabef-6ded-4119-9ae1-a9471c11de8b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:37.467278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:37.520373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fabef-6ded-4119-9ae1-a9471c11de8b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:37.564314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e59263-0668-4d23-9268-20213b7776e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:37.564945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e59263-0668-4d23-9268-20213b7776e5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.565273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e59263-0668-4d23-9268-20213b7776e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.565545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e59263-0668-4d23-9268-20213b7776e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.585519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e59263-0668-4d23-9268-20213b7776e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:37.586974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e59263-0668-4d23-9268-20213b7776e5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:37.602651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8988276-4d7c-46f6-96a9-afd280f1e214 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-72e59263-0668-4d23-9268-20213b7776e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:37.616386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:37.617681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:37.618666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:37.618666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:37.618666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:37.625371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 274/550] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:37.626440 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e59263-0668-4d23-9268-20213b7776e5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:37.651964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc053f7-1c13-4b46-b807-16ec2e5639e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:37.652522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc053f7-1c13-4b46-b807-16ec2e5639e5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.652800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc053f7-1c13-4b46-b807-16ec2e5639e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.653088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc053f7-1c13-4b46-b807-16ec2e5639e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.670303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc053f7-1c13-4b46-b807-16ec2e5639e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:37.671744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc053f7-1c13-4b46-b807-16ec2e5639e5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-adc053f7-1c13-4b46-b807-16ec2e5639e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:37.693496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:37.694725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:37.695771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:37.695771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:37.695771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:37.695771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 277/551] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:37.695997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbb8c5-7683-4f54-88fd-eb4daafc3c32 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.696540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbb8c5-7683-4f54-88fd-eb4daafc3c32 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.696540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbb8c5-7683-4f54-88fd-eb4daafc3c32 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.703026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 278/552] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:37.704342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbb8c5-7683-4f54-88fd-eb4daafc3c32 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:37.749439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:37.766364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:37.770229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e59263-0668-4d23-9268-20213b7776e5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:37.815106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dd784-f528-4a3e-aa1c-90cd386f120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:37.815712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dd784-f528-4a3e-aa1c-90cd386f120e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.815984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dd784-f528-4a3e-aa1c-90cd386f120e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.816247 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dd784-f528-4a3e-aa1c-90cd386f120e None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.816941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dd784-f528-4a3e-aa1c-90cd386f120e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:37.817285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dd784-f528-4a3e-aa1c-90cd386f120e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:37.818840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dd784-f528-4a3e-aa1c-90cd386f120e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:37.831435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 275/553] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:37.832701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcf970-65af-4346-b3e6-fb5a4e1c40b1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.833040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcf970-65af-4346-b3e6-fb5a4e1c40b1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.833333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcf970-65af-4346-b3e6-fb5a4e1c40b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.833669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbb8c5-7683-4f54-88fd-eb4daafc3c32 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:37.851983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-60dcf970-65af-4346-b3e6-fb5a4e1c40b1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:37.853186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-60dcf970-65af-4346-b3e6-fb5a4e1c40b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:37.871749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f9074-9aca-4ce7-a326-ad9e87099160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:37.872377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f9074-9aca-4ce7-a326-ad9e87099160 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.872632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f9074-9aca-4ce7-a326-ad9e87099160 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.872902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f9074-9aca-4ce7-a326-ad9e87099160 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.873573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f9074-9aca-4ce7-a326-ad9e87099160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:37.873873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f9074-9aca-4ce7-a326-ad9e87099160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:37.875876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f9074-9aca-4ce7-a326-ad9e87099160 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:37.887922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 279/554] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:37.889276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b597306-218b-4b82-b737-6c455f0eedde None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.889555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b597306-218b-4b82-b737-6c455f0eedde None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.889826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b597306-218b-4b82-b737-6c455f0eedde None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.893194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 280/555] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:37.893194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b597306-218b-4b82-b737-6c455f0eedde None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:37.930324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 276/556] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:37.931082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcf970-65af-4346-b3e6-fb5a4e1c40b1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.013394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.022784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b597306-218b-4b82-b737-6c455f0eedde None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.036301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.048152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcf970-65af-4346-b3e6-fb5a4e1c40b1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.068713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1137399-fcdd-45b1-babf-61e0eb41f274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.069893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1137399-fcdd-45b1-babf-61e0eb41f274 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.070167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1137399-fcdd-45b1-babf-61e0eb41f274 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.070432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1137399-fcdd-45b1-babf-61e0eb41f274 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.080777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1137399-fcdd-45b1-babf-61e0eb41f274 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.082536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1137399-fcdd-45b1-babf-61e0eb41f274 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.085591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a3e24-6f37-4d77-9aa2-b2accd7d953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.086285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a3e24-6f37-4d77-9aa2-b2accd7d953e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.086650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a3e24-6f37-4d77-9aa2-b2accd7d953e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.086982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a3e24-6f37-4d77-9aa2-b2accd7d953e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.094534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a3e24-6f37-4d77-9aa2-b2accd7d953e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.096042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a3e24-6f37-4d77-9aa2-b2accd7d953e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b1137399-fcdd-45b1-babf-61e0eb41f274 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:38.097397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:38.098326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:38.099229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:38.102341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 281/557] 10.222.0.78 () {56 vars in 1093 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:38.103650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff601a02-59b1-4d99-84e8-7e20d18d2028 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.104086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff601a02-59b1-4d99-84e8-7e20d18d2028 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.104526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff601a02-59b1-4d99-84e8-7e20d18d2028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-1f0a3e24-6f37-4d77-9aa2-b2accd7d953e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:38.112039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:38.113610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:48:38.114743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:38.120606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 277/558] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:38.120606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a3e24-6f37-4d77-9aa2-b2accd7d953e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.139006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ff601a02-59b1-4d99-84e8-7e20d18d2028 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:38.140970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ff601a02-59b1-4d99-84e8-7e20d18d2028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:38.237804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a3e24-6f37-4d77-9aa2-b2accd7d953e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.240011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 282/559] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:38.241015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff601a02-59b1-4d99-84e8-7e20d18d2028 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.280013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fab545-41ac-4e4d-bd05-de34a6e54b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.280624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fab545-41ac-4e4d-bd05-de34a6e54b3d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.280903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fab545-41ac-4e4d-bd05-de34a6e54b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.281250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fab545-41ac-4e4d-bd05-de34a6e54b3d None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.282422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fab545-41ac-4e4d-bd05-de34a6e54b3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:38.282729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fab545-41ac-4e4d-bd05-de34a6e54b3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.284315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fab545-41ac-4e4d-bd05-de34a6e54b3d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.307061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 278/560] 10.222.0.78 () {56 vars in 1133 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:38.307061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9844f-04bc-4e9e-97d9-cdd5c254ed5a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.307061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9844f-04bc-4e9e-97d9-cdd5c254ed5a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.307061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9844f-04bc-4e9e-97d9-cdd5c254ed5a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.309334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 279/561] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:38.310148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9844f-04bc-4e9e-97d9-cdd5c254ed5a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.321550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.342176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.358434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff601a02-59b1-4d99-84e8-7e20d18d2028 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.394817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24b903-cf17-460c-b8ac-c0271c817f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.395444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24b903-cf17-460c-b8ac-c0271c817f5b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.395720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24b903-cf17-460c-b8ac-c0271c817f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.395987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24b903-cf17-460c-b8ac-c0271c817f5b None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.396709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24b903-cf17-460c-b8ac-c0271c817f5b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:38.397010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24b903-cf17-460c-b8ac-c0271c817f5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.398471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24b903-cf17-460c-b8ac-c0271c817f5b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.418888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 283/562] 10.222.0.78 () {56 vars in 1135 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:38.419971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24b903-cf17-460c-b8ac-c0271c817f5b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.425788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9844f-04bc-4e9e-97d9-cdd5c254ed5a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.464624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9057b-724f-4db6-8417-c8abb715b0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.465269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9057b-724f-4db6-8417-c8abb715b0ab None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.465545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9057b-724f-4db6-8417-c8abb715b0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.465809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9057b-724f-4db6-8417-c8abb715b0ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.476627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9057b-724f-4db6-8417-c8abb715b0ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-19c9057b-724f-4db6-8417-c8abb715b0ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:38.481596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:38.482467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:38.483289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:38.485618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 280/563] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:38.486927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b569c8-add2-46aa-9c3f-acb21abf4077 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.487372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b569c8-add2-46aa-9c3f-acb21abf4077 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.487882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b569c8-add2-46aa-9c3f-acb21abf4077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.506706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-95b569c8-add2-46aa-9c3f-acb21abf4077 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:38.508078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-95b569c8-add2-46aa-9c3f-acb21abf4077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:38.533477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24b903-cf17-460c-b8ac-c0271c817f5b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.566873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9b187-3b5e-4d54-9cb0-f5b0de6fb4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.567485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9b187-3b5e-4d54-9cb0-f5b0de6fb4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.567756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9b187-3b5e-4d54-9cb0-f5b0de6fb4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.568027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9b187-3b5e-4d54-9cb0-f5b0de6fb4e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.572958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.578942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9b187-3b5e-4d54-9cb0-f5b0de6fb4e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.580389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9b187-3b5e-4d54-9cb0-f5b0de6fb4e2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.582982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.585215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 281/564] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:38.586317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b569c8-add2-46aa-9c3f-acb21abf4077 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-5cf9b187-3b5e-4d54-9cb0-f5b0de6fb4e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:38.596192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:38.597234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:38.598164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:38.598164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:38.598164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:38.598164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 284/565] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:38.598728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9b187-3b5e-4d54-9cb0-f5b0de6fb4e2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.698917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b569c8-add2-46aa-9c3f-acb21abf4077 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.708525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9b187-3b5e-4d54-9cb0-f5b0de6fb4e2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.735692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4f531-7222-4975-a801-629f517b4370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.736416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4f531-7222-4975-a801-629f517b4370 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.736693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4f531-7222-4975-a801-629f517b4370 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.736989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4f531-7222-4975-a801-629f517b4370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.741877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00335bdd-57b4-4651-ae67-b77a7e96ff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.742425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00335bdd-57b4-4651-ae67-b77a7e96ff4d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.742705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00335bdd-57b4-4651-ae67-b77a7e96ff4d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.742973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00335bdd-57b4-4651-ae67-b77a7e96ff4d None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.743627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00335bdd-57b4-4651-ae67-b77a7e96ff4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:38.743927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00335bdd-57b4-4651-ae67-b77a7e96ff4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.745350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00335bdd-57b4-4651-ae67-b77a7e96ff4d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.758449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 285/566] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:38.758449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00335bdd-57b4-4651-ae67-b77a7e96ff4d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.811823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.817334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.818299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4f531-7222-4975-a801-629f517b4370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.819989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4f531-7222-4975-a801-629f517b4370 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.862238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00335bdd-57b4-4651-ae67-b77a7e96ff4d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.896689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185467a-036c-4d4e-a3e9-fef9c9fe9581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.897745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185467a-036c-4d4e-a3e9-fef9c9fe9581 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.898030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185467a-036c-4d4e-a3e9-fef9c9fe9581 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.898301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185467a-036c-4d4e-a3e9-fef9c9fe9581 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.908393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185467a-036c-4d4e-a3e9-fef9c9fe9581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3185467a-036c-4d4e-a3e9-fef9c9fe9581 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:38.913630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:38.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:38.915379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 286/567] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:38.920158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185467a-036c-4d4e-a3e9-fef9c9fe9581 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.922536 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 282/568] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:38.923352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4f531-7222-4975-a801-629f517b4370 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.018999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185467a-036c-4d4e-a3e9-fef9c9fe9581 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.021534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.025248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4f531-7222-4975-a801-629f517b4370 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.037019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.055621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29daadb-f483-4705-ad5b-251988acd7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.056225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29daadb-f483-4705-ad5b-251988acd7fd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.056504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29daadb-f483-4705-ad5b-251988acd7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.056766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29daadb-f483-4705-ad5b-251988acd7fd None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.057418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29daadb-f483-4705-ad5b-251988acd7fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:39.057718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29daadb-f483-4705-ad5b-251988acd7fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.059072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29daadb-f483-4705-ad5b-251988acd7fd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.061020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad047ca5-84af-437c-a40d-d201b098408e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.061667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad047ca5-84af-437c-a40d-d201b098408e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.061939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad047ca5-84af-437c-a40d-d201b098408e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.062202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad047ca5-84af-437c-a40d-d201b098408e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.071561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 287/569] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.072366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29daadb-f483-4705-ad5b-251988acd7fd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.073449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad047ca5-84af-437c-a40d-d201b098408e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.074882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad047ca5-84af-437c-a40d-d201b098408e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ad047ca5-84af-437c-a40d-d201b098408e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:39.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:39.090682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:39.091569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:39.091569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:39.091569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:39.091569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 283/570] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:39.091803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad047ca5-84af-437c-a40d-d201b098408e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.174231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29daadb-f483-4705-ad5b-251988acd7fd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.190363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad047ca5-84af-437c-a40d-d201b098408e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.207185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6252da-3566-4fa6-a8ee-81f22f3ce668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.207811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6252da-3566-4fa6-a8ee-81f22f3ce668 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.208090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6252da-3566-4fa6-a8ee-81f22f3ce668 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.208361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6252da-3566-4fa6-a8ee-81f22f3ce668 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.218150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6252da-3566-4fa6-a8ee-81f22f3ce668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:39.223471 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fac131-f296-4dc8-a254-a4c6d3061b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-bc6252da-3566-4fa6-a8ee-81f22f3ce668 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:39.223618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:39.224495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:39.225352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:39.225968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fac131-f296-4dc8-a254-a4c6d3061b39 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.225968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fac131-f296-4dc8-a254-a4c6d3061b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.226103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 288/571] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:39.226257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fac131-f296-4dc8-a254-a4c6d3061b39 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.226257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fac131-f296-4dc8-a254-a4c6d3061b39 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:39.226389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6252da-3566-4fa6-a8ee-81f22f3ce668 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.226456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fac131-f296-4dc8-a254-a4c6d3061b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.227125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fac131-f296-4dc8-a254-a4c6d3061b39 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.237598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 284/572] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.238396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fac131-f296-4dc8-a254-a4c6d3061b39 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.254327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.271003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.331261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6252da-3566-4fa6-a8ee-81f22f3ce668 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.343068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fac131-f296-4dc8-a254-a4c6d3061b39 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.365330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d1c9-58e7-4207-9c79-86b5a77bb624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.365942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d1c9-58e7-4207-9c79-86b5a77bb624 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.366218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d1c9-58e7-4207-9c79-86b5a77bb624 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.366485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d1c9-58e7-4207-9c79-86b5a77bb624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.376214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02d1c9-58e7-4207-9c79-86b5a77bb624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.377022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fe0e4-06d8-4fbe-9b0a-6731a32bf14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.377673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fe0e4-06d8-4fbe-9b0a-6731a32bf14f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.377767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02d1c9-58e7-4207-9c79-86b5a77bb624 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.377953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fe0e4-06d8-4fbe-9b0a-6731a32bf14f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.378224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fe0e4-06d8-4fbe-9b0a-6731a32bf14f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.384822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fe0e4-06d8-4fbe-9b0a-6731a32bf14f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.386356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fe0e4-06d8-4fbe-9b0a-6731a32bf14f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8e02d1c9-58e7-4207-9c79-86b5a77bb624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:39.392257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:39.393206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:39.394063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:39.394063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:39.394063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:39.394063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 289/573] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:39.394292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d1c9-58e7-4207-9c79-86b5a77bb624 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d58fe0e4-06d8-4fbe-9b0a-6731a32bf14f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:39.397479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:39.398304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:39.399162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:39.399162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:39.399162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:39.399162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:39.399162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:39.399162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:39.399162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:39.399162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:39.399162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 285/574] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:39.399667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fe0e4-06d8-4fbe-9b0a-6731a32bf14f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.468136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.491210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.499429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fe0e4-06d8-4fbe-9b0a-6731a32bf14f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.501050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d1c9-58e7-4207-9c79-86b5a77bb624 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.534118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47a5bb-9fb4-4490-a82a-8b1eef9f3f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.534293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3c586-1f95-49b7-9fae-dbc89b77e219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.534642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47a5bb-9fb4-4490-a82a-8b1eef9f3f9d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.534740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3c586-1f95-49b7-9fae-dbc89b77e219 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.534913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47a5bb-9fb4-4490-a82a-8b1eef9f3f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.535005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3c586-1f95-49b7-9fae-dbc89b77e219 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.535178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47a5bb-9fb4-4490-a82a-8b1eef9f3f9d None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.535277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3c586-1f95-49b7-9fae-dbc89b77e219 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.535800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47a5bb-9fb4-4490-a82a-8b1eef9f3f9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:39.535902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3c586-1f95-49b7-9fae-dbc89b77e219 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:39.536120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3c586-1f95-49b7-9fae-dbc89b77e219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.536219 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47a5bb-9fb4-4490-a82a-8b1eef9f3f9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.537520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3c586-1f95-49b7-9fae-dbc89b77e219 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.537654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47a5bb-9fb4-4490-a82a-8b1eef9f3f9d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.545431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 286/575] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.546215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3c586-1f95-49b7-9fae-dbc89b77e219 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.551190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 290/576] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.551190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47a5bb-9fb4-4490-a82a-8b1eef9f3f9d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.646521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3c586-1f95-49b7-9fae-dbc89b77e219 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.647645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47a5bb-9fb4-4490-a82a-8b1eef9f3f9d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.680019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50d15d-163d-405f-b025-d30229518dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.680639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50d15d-163d-405f-b025-d30229518dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.680920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50d15d-163d-405f-b025-d30229518dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.681095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb921c7-e5cd-4d10-b452-d24b3964bca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.681257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50d15d-163d-405f-b025-d30229518dd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.681693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb921c7-e5cd-4d10-b452-d24b3964bca6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.681964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb921c7-e5cd-4d10-b452-d24b3964bca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.682228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb921c7-e5cd-4d10-b452-d24b3964bca6 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.682850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb921c7-e5cd-4d10-b452-d24b3964bca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:39.683154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb921c7-e5cd-4d10-b452-d24b3964bca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.683281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.684633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb921c7-e5cd-4d10-b452-d24b3964bca6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.694000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50d15d-163d-405f-b025-d30229518dd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.695701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50d15d-163d-405f-b025-d30229518dd9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.696456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 291/577] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.697293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb921c7-e5cd-4d10-b452-d24b3964bca6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3f50d15d-163d-405f-b025-d30229518dd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:39.710750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:39.711823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:39.713026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:39.713026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:39.713026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:39.713251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.716724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 287/578] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:39.717601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50d15d-163d-405f-b025-d30229518dd9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.799795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb921c7-e5cd-4d10-b452-d24b3964bca6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.817388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50d15d-163d-405f-b025-d30229518dd9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.832912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa5e4-1565-4eb7-8ac7-51d3d7436c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.833544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa5e4-1565-4eb7-8ac7-51d3d7436c67 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.833824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa5e4-1565-4eb7-8ac7-51d3d7436c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.834095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa5e4-1565-4eb7-8ac7-51d3d7436c67 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.834779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa5e4-1565-4eb7-8ac7-51d3d7436c67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:39.835081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa5e4-1565-4eb7-8ac7-51d3d7436c67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.836554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa5e4-1565-4eb7-8ac7-51d3d7436c67 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.848734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 292/579] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.849673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa5e4-1565-4eb7-8ac7-51d3d7436c67 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.852150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5b9cf-f5e6-4499-8f30-ba78b49ec1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.852776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5b9cf-f5e6-4499-8f30-ba78b49ec1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.853061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5b9cf-f5e6-4499-8f30-ba78b49ec1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.853357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5b9cf-f5e6-4499-8f30-ba78b49ec1e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.864131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5b9cf-f5e6-4499-8f30-ba78b49ec1e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.865601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5b9cf-f5e6-4499-8f30-ba78b49ec1e6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-4cd5b9cf-f5e6-4499-8f30-ba78b49ec1e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:39.879945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:39.881026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:39.882080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:39.882080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:39.882080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:39.882080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 288/580] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:39.882080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5b9cf-f5e6-4499-8f30-ba78b49ec1e6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.923104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.923617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.953964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa5e4-1565-4eb7-8ac7-51d3d7436c67 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.985102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5b9cf-f5e6-4499-8f30-ba78b49ec1e6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.986982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11691e2f-d079-4038-b00f-c3eba437ca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.987600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11691e2f-d079-4038-b00f-c3eba437ca55 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.987877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11691e2f-d079-4038-b00f-c3eba437ca55 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.988144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11691e2f-d079-4038-b00f-c3eba437ca55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.995030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11691e2f-d079-4038-b00f-c3eba437ca55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.996479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11691e2f-d079-4038-b00f-c3eba437ca55 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-11691e2f-d079-4038-b00f-c3eba437ca55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:40.009484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:40.010622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 293/581] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:40.011986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11691e2f-d079-4038-b00f-c3eba437ca55 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.022855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec91064-f03d-4e36-9c03-6af5c0a56467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.023481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec91064-f03d-4e36-9c03-6af5c0a56467 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.023767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec91064-f03d-4e36-9c03-6af5c0a56467 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.024038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec91064-f03d-4e36-9c03-6af5c0a56467 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.030734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec91064-f03d-4e36-9c03-6af5c0a56467 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.032203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec91064-f03d-4e36-9c03-6af5c0a56467 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-0ec91064-f03d-4e36-9c03-6af5c0a56467 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:40.043279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:40.044315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 289/582] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:40.045280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec91064-f03d-4e36-9c03-6af5c0a56467 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.114056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11691e2f-d079-4038-b00f-c3eba437ca55 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.142797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.143720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.148416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec91064-f03d-4e36-9c03-6af5c0a56467 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.149008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe650a-d9a3-4a90-a722-4a28e3f2310a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.149642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe650a-d9a3-4a90-a722-4a28e3f2310a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.149922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe650a-d9a3-4a90-a722-4a28e3f2310a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.150226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe650a-d9a3-4a90-a722-4a28e3f2310a None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.150858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe650a-d9a3-4a90-a722-4a28e3f2310a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:40.151157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe650a-d9a3-4a90-a722-4a28e3f2310a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.152531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe650a-d9a3-4a90-a722-4a28e3f2310a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.164505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 294/583] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:40.165403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe650a-d9a3-4a90-a722-4a28e3f2310a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.185125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac77f2-a4d7-41be-8e6b-f051e6b90b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.185789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac77f2-a4d7-41be-8e6b-f051e6b90b25 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.186090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac77f2-a4d7-41be-8e6b-f051e6b90b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.186362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac77f2-a4d7-41be-8e6b-f051e6b90b25 None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/f9b88c495fcb47a289a6a45276c891e0/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.201203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac77f2-a4d7-41be-8e6b-f051e6b90b25 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=f9b88c495fcb47a289a6a45276c891e0, role_id=fd7dbf5fe1ae4cdf8a10f67c6921d3da)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.202612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac77f2-a4d7-41be-8e6b-f051e6b90b25 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.216755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 290/584] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:40 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/f9b88c495fcb47a289a6a45276c891e0/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:40.217574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac77f2-a4d7-41be-8e6b-f051e6b90b25 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.284169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe650a-d9a3-4a90-a722-4a28e3f2310a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.324027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b71bb-5d45-4b2b-8179-876439270e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.324706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b71bb-5d45-4b2b-8179-876439270e46 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.325005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b71bb-5d45-4b2b-8179-876439270e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.325295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b71bb-5d45-4b2b-8179-876439270e46 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.325956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b71bb-5d45-4b2b-8179-876439270e46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:40.326263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b71bb-5d45-4b2b-8179-876439270e46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.327668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b71bb-5d45-4b2b-8179-876439270e46 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.339661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac77f2-a4d7-41be-8e6b-f051e6b90b25 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.345209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 295/585] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:40.345209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b71bb-5d45-4b2b-8179-876439270e46 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.380889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b0af1-f572-45f6-b59d-4b69e6f8c6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.381563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b0af1-f572-45f6-b59d-4b69e6f8c6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.381870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b0af1-f572-45f6-b59d-4b69e6f8c6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.382239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b0af1-f572-45f6-b59d-4b69e6f8c6d2 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.382980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b0af1-f572-45f6-b59d-4b69e6f8c6d2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:40.383288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b0af1-f572-45f6-b59d-4b69e6f8c6d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.384794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b0af1-f572-45f6-b59d-4b69e6f8c6d2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.402934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 291/586] 10.222.0.78 () {56 vars in 1137 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:40.403622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b0af1-f572-45f6-b59d-4b69e6f8c6d2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.408566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.412254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.464540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b71bb-5d45-4b2b-8179-876439270e46 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.507086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac716e-8bb9-4127-9d49-4620855497b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.507857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac716e-8bb9-4127-9d49-4620855497b7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.508444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac716e-8bb9-4127-9d49-4620855497b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.508869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac716e-8bb9-4127-9d49-4620855497b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.509751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac716e-8bb9-4127-9d49-4620855497b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:40.510225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac716e-8bb9-4127-9d49-4620855497b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.512366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac716e-8bb9-4127-9d49-4620855497b7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.513492 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b0af1-f572-45f6-b59d-4b69e6f8c6d2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.532758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 296/587] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:40.532758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac716e-8bb9-4127-9d49-4620855497b7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.550444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f83fa5-d703-4d0f-99a7-72acc69278d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.551048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f83fa5-d703-4d0f-99a7-72acc69278d0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.551326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f83fa5-d703-4d0f-99a7-72acc69278d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.551591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f83fa5-d703-4d0f-99a7-72acc69278d0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.558770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f83fa5-d703-4d0f-99a7-72acc69278d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.560184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f83fa5-d703-4d0f-99a7-72acc69278d0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-e4f83fa5-d703-4d0f-99a7-72acc69278d0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:40.572599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:40.573609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 292/588] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:40.574565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f83fa5-d703-4d0f-99a7-72acc69278d0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.646362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac716e-8bb9-4127-9d49-4620855497b7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.650325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.658902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.689978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bfd56-fbe0-41de-9650-f07cc09aff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.690608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bfd56-fbe0-41de-9650-f07cc09aff33 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.690785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f83fa5-d703-4d0f-99a7-72acc69278d0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.690894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bfd56-fbe0-41de-9650-f07cc09aff33 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.691146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bfd56-fbe0-41de-9650-f07cc09aff33 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.701028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bfd56-fbe0-41de-9650-f07cc09aff33 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.702945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bfd56-fbe0-41de-9650-f07cc09aff33 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a20bfd56-fbe0-41de-9650-f07cc09aff33 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:40.718356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:40.720053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:40.720965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 297/589] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:40.722280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bfd56-fbe0-41de-9650-f07cc09aff33 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.732242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb28e24-cfdf-4448-8183-861dc96a1204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.733159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb28e24-cfdf-4448-8183-861dc96a1204 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.733443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb28e24-cfdf-4448-8183-861dc96a1204 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.733793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb28e24-cfdf-4448-8183-861dc96a1204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.745867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb28e24-cfdf-4448-8183-861dc96a1204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.747290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb28e24-cfdf-4448-8183-861dc96a1204 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-0eb28e24-cfdf-4448-8183-861dc96a1204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:40.762410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:40.763388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:40.764266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:40.764266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:40.764266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:40.764266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 293/590] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:40.764958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb28e24-cfdf-4448-8183-861dc96a1204 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.830251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bfd56-fbe0-41de-9650-f07cc09aff33 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.868224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1516440-fbe2-47e3-a07d-634a6d59581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.868818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1516440-fbe2-47e3-a07d-634a6d59581d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.869120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1516440-fbe2-47e3-a07d-634a6d59581d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.869393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1516440-fbe2-47e3-a07d-634a6d59581d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.870108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1516440-fbe2-47e3-a07d-634a6d59581d None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=c53b9bcd675940c19febbc8fefb1f28b) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:40.870419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1516440-fbe2-47e3-a07d-634a6d59581d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.871852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1516440-fbe2-47e3-a07d-634a6d59581d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.874248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb28e24-cfdf-4448-8183-861dc96a1204 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.877981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 298/591] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/role_assignments?user.id=c53b9bcd675940c19febbc8fefb1f28b&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:40.878775 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1516440-fbe2-47e3-a07d-634a6d59581d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.907671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.921434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.932231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4b1f0-bc8c-4b7d-b457-c110f20983e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.933396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4b1f0-bc8c-4b7d-b457-c110f20983e4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.933949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4b1f0-bc8c-4b7d-b457-c110f20983e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.934529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4b1f0-bc8c-4b7d-b457-c110f20983e4 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.935480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4b1f0-bc8c-4b7d-b457-c110f20983e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:40.935990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4b1f0-bc8c-4b7d-b457-c110f20983e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.938635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4b1f0-bc8c-4b7d-b457-c110f20983e4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.961275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 294/592] 10.222.0.78 () {56 vars in 1137 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:40.962702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4b1f0-bc8c-4b7d-b457-c110f20983e4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.999033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1516440-fbe2-47e3-a07d-634a6d59581d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.037942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae66f8-0391-4797-8859-033b73539c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.038671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae66f8-0391-4797-8859-033b73539c8b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.038949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae66f8-0391-4797-8859-033b73539c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.039326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae66f8-0391-4797-8859-033b73539c8b None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.040055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae66f8-0391-4797-8859-033b73539c8b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:41.040356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae66f8-0391-4797-8859-033b73539c8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.041886 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae66f8-0391-4797-8859-033b73539c8b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.064489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 299/593] 10.222.0.78 () {56 vars in 1139 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:41.069123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae66f8-0391-4797-8859-033b73539c8b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.106621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4b1f0-bc8c-4b7d-b457-c110f20983e4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.160838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25a6e2-c662-4943-a7c1-5eaa12eeb2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.161539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25a6e2-c662-4943-a7c1-5eaa12eeb2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.161629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25a6e2-c662-4943-a7c1-5eaa12eeb2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.161918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25a6e2-c662-4943-a7c1-5eaa12eeb2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.162600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25a6e2-c662-4943-a7c1-5eaa12eeb2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:41.163347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25a6e2-c662-4943-a7c1-5eaa12eeb2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.165018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25a6e2-c662-4943-a7c1-5eaa12eeb2a6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.178347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:41.185180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 295/594] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:41.185180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25a6e2-c662-4943-a7c1-5eaa12eeb2a6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.193932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae66f8-0391-4797-8859-033b73539c8b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.231860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:41.233914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215aa1ad-43a8-4f44-a5fd-e674cac13958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.236035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215aa1ad-43a8-4f44-a5fd-e674cac13958 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.236496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215aa1ad-43a8-4f44-a5fd-e674cac13958 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.236914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215aa1ad-43a8-4f44-a5fd-e674cac13958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.254061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215aa1ad-43a8-4f44-a5fd-e674cac13958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.255795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215aa1ad-43a8-4f44-a5fd-e674cac13958 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-215aa1ad-43a8-4f44-a5fd-e674cac13958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:41.277065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:41.278240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:41.279143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:41.279143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:41.279143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:41.286628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 300/595] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:41.287675 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215aa1ad-43a8-4f44-a5fd-e674cac13958 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.320890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25a6e2-c662-4943-a7c1-5eaa12eeb2a6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.360606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f02187-6fc2-4f36-96b7-f20417f12d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.361404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f02187-6fc2-4f36-96b7-f20417f12d08 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.361765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f02187-6fc2-4f36-96b7-f20417f12d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.362040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f02187-6fc2-4f36-96b7-f20417f12d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.373379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f02187-6fc2-4f36-96b7-f20417f12d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.374823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f02187-6fc2-4f36-96b7-f20417f12d08 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a9f02187-6fc2-4f36-96b7-f20417f12d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:41.390343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:41.391546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:41.392621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:41.392621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:41.392621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:41.392621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 296/596] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:41.397109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f02187-6fc2-4f36-96b7-f20417f12d08 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.408888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215aa1ad-43a8-4f44-a5fd-e674cac13958 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.446299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17102922-469c-41bf-bbaf-0f6d4c3889f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.446845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17102922-469c-41bf-bbaf-0f6d4c3889f4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.447124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17102922-469c-41bf-bbaf-0f6d4c3889f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.447394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17102922-469c-41bf-bbaf-0f6d4c3889f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.455893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17102922-469c-41bf-bbaf-0f6d4c3889f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.457487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17102922-469c-41bf-bbaf-0f6d4c3889f4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-17102922-469c-41bf-bbaf-0f6d4c3889f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:41.473890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:41.474958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 301/597] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:41.476009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17102922-469c-41bf-bbaf-0f6d4c3889f4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.480751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:41.498094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:41.516723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f02187-6fc2-4f36-96b7-f20417f12d08 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.561877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3874a-c55a-40ca-86e3-4d0c7305efd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.563556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3874a-c55a-40ca-86e3-4d0c7305efd6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.564014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3874a-c55a-40ca-86e3-4d0c7305efd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.564427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3874a-c55a-40ca-86e3-4d0c7305efd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.565321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3874a-c55a-40ca-86e3-4d0c7305efd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:41.565785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3874a-c55a-40ca-86e3-4d0c7305efd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.568128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3874a-c55a-40ca-86e3-4d0c7305efd6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.585035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 297/598] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:41.586094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3874a-c55a-40ca-86e3-4d0c7305efd6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.617250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17102922-469c-41bf-bbaf-0f6d4c3889f4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.667096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f3a72-5b72-4ad2-baa5-1fc9885ec56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.667709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f3a72-5b72-4ad2-baa5-1fc9885ec56f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.668220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f3a72-5b72-4ad2-baa5-1fc9885ec56f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.668506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f3a72-5b72-4ad2-baa5-1fc9885ec56f None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.669173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f3a72-5b72-4ad2-baa5-1fc9885ec56f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:41.669502 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f3a72-5b72-4ad2-baa5-1fc9885ec56f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.671915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f3a72-5b72-4ad2-baa5-1fc9885ec56f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.689778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 302/599] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:41.693964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f3a72-5b72-4ad2-baa5-1fc9885ec56f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.716810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3874a-c55a-40ca-86e3-4d0c7305efd6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.760419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07553c-24c1-416d-9a9b-5629a2c86319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.761499 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07553c-24c1-416d-9a9b-5629a2c86319 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.762336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07553c-24c1-416d-9a9b-5629a2c86319 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.763173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07553c-24c1-416d-9a9b-5629a2c86319 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.764041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07553c-24c1-416d-9a9b-5629a2c86319 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:41.764651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07553c-24c1-416d-9a9b-5629a2c86319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.766674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07553c-24c1-416d-9a9b-5629a2c86319 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.783579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 298/600] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:41.784647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07553c-24c1-416d-9a9b-5629a2c86319 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.785821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:41.803771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:41.833600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f3a72-5b72-4ad2-baa5-1fc9885ec56f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.875248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb5b92-cef1-4e1b-ba11-bac517c497b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.875878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb5b92-cef1-4e1b-ba11-bac517c497b0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.876152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb5b92-cef1-4e1b-ba11-bac517c497b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.876415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb5b92-cef1-4e1b-ba11-bac517c497b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.885761 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb5b92-cef1-4e1b-ba11-bac517c497b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.887212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb5b92-cef1-4e1b-ba11-bac517c497b0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8beb5b92-cef1-4e1b-ba11-bac517c497b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:41.909200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:41.910353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 303/601] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:41.911272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb5b92-cef1-4e1b-ba11-bac517c497b0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.929134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07553c-24c1-416d-9a9b-5629a2c86319 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.964165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34064ca9-11e9-406e-bd17-d547ea482be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.964796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34064ca9-11e9-406e-bd17-d547ea482be1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.965069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34064ca9-11e9-406e-bd17-d547ea482be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.965360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34064ca9-11e9-406e-bd17-d547ea482be1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.973159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34064ca9-11e9-406e-bd17-d547ea482be1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.974533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34064ca9-11e9-406e-bd17-d547ea482be1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-34064ca9-11e9-406e-bd17-d547ea482be1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:41.986793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:41.987710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:41.988638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:41.988638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:41.988638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:41.988638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:41.988638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:41.988638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:41.988638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:41.988638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:41.992472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 299/602] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:41.993324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34064ca9-11e9-406e-bd17-d547ea482be1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.033208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb5b92-cef1-4e1b-ba11-bac517c497b0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.061356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.062864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.078554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4fb63-9317-4e56-9395-d83b3e3b4286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.080157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4fb63-9317-4e56-9395-d83b3e3b4286 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.080649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4fb63-9317-4e56-9395-d83b3e3b4286 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.081176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4fb63-9317-4e56-9395-d83b3e3b4286 None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/97e64ee3c8ee48c2ad8f351fb2a78bc8/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.098519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4fb63-9317-4e56-9395-d83b3e3b4286 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=97e64ee3c8ee48c2ad8f351fb2a78bc8, role_id=fd7dbf5fe1ae4cdf8a10f67c6921d3da)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.098928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34064ca9-11e9-406e-bd17-d547ea482be1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.100630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4fb63-9317-4e56-9395-d83b3e3b4286 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.116324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 304/603] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:41 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/97e64ee3c8ee48c2ad8f351fb2a78bc8/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:42.117513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4fb63-9317-4e56-9395-d83b3e3b4286 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.134421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdbb1c-41d4-44ed-9c56-3f304b446e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.135037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdbb1c-41d4-44ed-9c56-3f304b446e03 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.135309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdbb1c-41d4-44ed-9c56-3f304b446e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.135579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdbb1c-41d4-44ed-9c56-3f304b446e03 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.136271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cdbb1c-41d4-44ed-9c56-3f304b446e03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:42.136572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cdbb1c-41d4-44ed-9c56-3f304b446e03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.138027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cdbb1c-41d4-44ed-9c56-3f304b446e03 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.151918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 300/604] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:42.169732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdbb1c-41d4-44ed-9c56-3f304b446e03 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.271391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4fb63-9317-4e56-9395-d83b3e3b4286 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.329263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418fb91-2e31-46ad-a77c-8937b6f6eac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.329263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418fb91-2e31-46ad-a77c-8937b6f6eac9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.329578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418fb91-2e31-46ad-a77c-8937b6f6eac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.333114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418fb91-2e31-46ad-a77c-8937b6f6eac9 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.333114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418fb91-2e31-46ad-a77c-8937b6f6eac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:42.333114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418fb91-2e31-46ad-a77c-8937b6f6eac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.333421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418fb91-2e31-46ad-a77c-8937b6f6eac9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.353151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 305/605] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:42.354825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d278f-a06f-4a99-b18b-f0c6d144708e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.356841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdbb1c-41d4-44ed-9c56-3f304b446e03 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.357255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d278f-a06f-4a99-b18b-f0c6d144708e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.357637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d278f-a06f-4a99-b18b-f0c6d144708e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.360733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 306/606] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:42.362473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d278f-a06f-4a99-b18b-f0c6d144708e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.389686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.398796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5a4d0-1390-475e-815d-8f3452e43460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.399676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5a4d0-1390-475e-815d-8f3452e43460 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.400084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5a4d0-1390-475e-815d-8f3452e43460 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.400497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5a4d0-1390-475e-815d-8f3452e43460 None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/77a47e77cc1942618a3584862042c5f7/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.410450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.416934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5a4d0-1390-475e-815d-8f3452e43460 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=77a47e77cc1942618a3584862042c5f7, role_id=fd7dbf5fe1ae4cdf8a10f67c6921d3da)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.418666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5a4d0-1390-475e-815d-8f3452e43460 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.445131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 301/607] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:42 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/77a47e77cc1942618a3584862042c5f7/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:48:42.445131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d7fec-a888-4ee3-a79d-72d2d61f0cf8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.445131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d7fec-a888-4ee3-a79d-72d2d61f0cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.445131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d7fec-a888-4ee3-a79d-72d2d61f0cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.461165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6e1d7fec-a888-4ee3-a79d-72d2d61f0cf8 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:42.462437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6e1d7fec-a888-4ee3-a79d-72d2d61f0cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:42.539866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d278f-a06f-4a99-b18b-f0c6d144708e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.554509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 302/608] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:42.610417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5f993-3b47-4a86-8f43-6dd4081771b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.611149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5f993-3b47-4a86-8f43-6dd4081771b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.611433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5f993-3b47-4a86-8f43-6dd4081771b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.611705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5f993-3b47-4a86-8f43-6dd4081771b0 None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/1a5d841380c243209f55314b545ff56f/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.634612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5f993-3b47-4a86-8f43-6dd4081771b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=1a5d841380c243209f55314b545ff56f, role_id=fd7dbf5fe1ae4cdf8a10f67c6921d3da)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.636105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5f993-3b47-4a86-8f43-6dd4081771b0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.664473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 307/609] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:42 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/1a5d841380c243209f55314b545ff56f/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:42.870967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d7fec-a888-4ee3-a79d-72d2d61f0cf8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.917312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20d791-ab24-48d0-bc62-24ba2da931e0 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.917660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20d791-ab24-48d0-bc62-24ba2da931e0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.917865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20d791-ab24-48d0-bc62-24ba2da931e0 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.919470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 308/610] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:42.924582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25347504-faac-4e78-8bee-2bbc572d19c5 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.924867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25347504-faac-4e78-8bee-2bbc572d19c5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.925166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25347504-faac-4e78-8bee-2bbc572d19c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.945694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-25347504-faac-4e78-8bee-2bbc572d19c5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:42.947179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-25347504-faac-4e78-8bee-2bbc572d19c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:42.979507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d7fec-a888-4ee3-a79d-72d2d61f0cf8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.989188 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.020068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550f65e-55b8-45dc-9580-7e6bb01cecbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.021154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550f65e-55b8-45dc-9580-7e6bb01cecbd None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.021890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550f65e-55b8-45dc-9580-7e6bb01cecbd None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.022445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550f65e-55b8-45dc-9580-7e6bb01cecbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.029689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 309/611] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:43.037103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550f65e-55b8-45dc-9580-7e6bb01cecbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-8550f65e-55b8-45dc-9580-7e6bb01cecbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:43.042332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:43.043386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:43.044361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:43.051344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 303/612] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:43.058180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25347504-faac-4e78-8bee-2bbc572d19c5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.087996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.215916 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25347504-faac-4e78-8bee-2bbc572d19c5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.252456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9627b08-9c7a-482a-a441-40ae8fe9b7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.253154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9627b08-9c7a-482a-a441-40ae8fe9b7bc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.253443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9627b08-9c7a-482a-a441-40ae8fe9b7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.253714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9627b08-9c7a-482a-a441-40ae8fe9b7bc None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.254382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9627b08-9c7a-482a-a441-40ae8fe9b7bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:43.254681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9627b08-9c7a-482a-a441-40ae8fe9b7bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.256053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9627b08-9c7a-482a-a441-40ae8fe9b7bc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.265134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 310/613] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:43.273445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550f65e-55b8-45dc-9580-7e6bb01cecbd None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.339187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9627b08-9c7a-482a-a441-40ae8fe9b7bc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.426960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550f65e-55b8-45dc-9580-7e6bb01cecbd None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.488346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.492968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6b460-68f1-42e6-8614-1ae23cd52680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.493632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6b460-68f1-42e6-8614-1ae23cd52680 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.493905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6b460-68f1-42e6-8614-1ae23cd52680 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.494171 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6b460-68f1-42e6-8614-1ae23cd52680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.518057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6b460-68f1-42e6-8614-1ae23cd52680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.519567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6b460-68f1-42e6-8614-1ae23cd52680 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.536564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9627b08-9c7a-482a-a441-40ae8fe9b7bc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7ef6b460-68f1-42e6-8614-1ae23cd52680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:43.550582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:43.551531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:43.552415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:43.552415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:43.552415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:43.552415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 304/614] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:43.557438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6b460-68f1-42e6-8614-1ae23cd52680 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.586741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7436cb-b108-4eae-9f3a-9d23983696d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.587391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7436cb-b108-4eae-9f3a-9d23983696d4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.587663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7436cb-b108-4eae-9f3a-9d23983696d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.587926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7436cb-b108-4eae-9f3a-9d23983696d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.598455 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.606406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7436cb-b108-4eae-9f3a-9d23983696d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-9c7436cb-b108-4eae-9f3a-9d23983696d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:43.614802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:43.615726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:43.616553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:43.617204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 311/615] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:43.625752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7436cb-b108-4eae-9f3a-9d23983696d4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.706598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6b460-68f1-42e6-8614-1ae23cd52680 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.745420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380465bb-8977-4264-8f3a-297ca10006ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.748144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380465bb-8977-4264-8f3a-297ca10006ba None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.748646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380465bb-8977-4264-8f3a-297ca10006ba None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.749950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380465bb-8977-4264-8f3a-297ca10006ba None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.750871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380465bb-8977-4264-8f3a-297ca10006ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:43.751583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380465bb-8977-4264-8f3a-297ca10006ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.754120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380465bb-8977-4264-8f3a-297ca10006ba None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.771660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 305/616] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:43.778706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380465bb-8977-4264-8f3a-297ca10006ba None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.807220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7436cb-b108-4eae-9f3a-9d23983696d4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.842892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661d84b-e3a3-4598-9343-4bd8e860a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.843493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661d84b-e3a3-4598-9343-4bd8e860a20c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.843940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661d84b-e3a3-4598-9343-4bd8e860a20c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.844208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661d84b-e3a3-4598-9343-4bd8e860a20c None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.844858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661d84b-e3a3-4598-9343-4bd8e860a20c None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:43.845182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661d84b-e3a3-4598-9343-4bd8e860a20c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.846552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661d84b-e3a3-4598-9343-4bd8e860a20c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.855122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.855366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 312/617] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:43.862201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661d84b-e3a3-4598-9343-4bd8e860a20c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.947946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.972615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661d84b-e3a3-4598-9343-4bd8e860a20c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.000863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380465bb-8977-4264-8f3a-297ca10006ba None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.035313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce488d-6e8f-4f57-b70a-91db68ba084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.036191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce488d-6e8f-4f57-b70a-91db68ba084b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.036637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce488d-6e8f-4f57-b70a-91db68ba084b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.037062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce488d-6e8f-4f57-b70a-91db68ba084b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.039664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8514d-9ad1-4da8-955f-c2108707345c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.040518 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8514d-9ad1-4da8-955f-c2108707345c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.041000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8514d-9ad1-4da8-955f-c2108707345c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.041519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8514d-9ad1-4da8-955f-c2108707345c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.050935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8514d-9ad1-4da8-955f-c2108707345c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.052661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8514d-9ad1-4da8-955f-c2108707345c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.063773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce488d-6e8f-4f57-b70a-91db68ba084b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.065272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce488d-6e8f-4f57-b70a-91db68ba084b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d9e8514d-9ad1-4da8-955f-c2108707345c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:44.067137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:44.068023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:44.069036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:44.072071 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 306/618] 10.222.0.78 () {56 vars in 1093 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:44.077165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8514d-9ad1-4da8-955f-c2108707345c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-89ce488d-6e8f-4f57-b70a-91db68ba084b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:44.091713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:44.092876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:44.094034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:44.094034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:44.094034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:44.097401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 313/619] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:44.098422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce488d-6e8f-4f57-b70a-91db68ba084b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.188646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8514d-9ad1-4da8-955f-c2108707345c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.227676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.248438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154bb2-f7ff-46eb-9ef9-76c1c5f16636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.249430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154bb2-f7ff-46eb-9ef9-76c1c5f16636 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.249972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154bb2-f7ff-46eb-9ef9-76c1c5f16636 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.250993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154bb2-f7ff-46eb-9ef9-76c1c5f16636 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.252639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd154bb2-f7ff-46eb-9ef9-76c1c5f16636 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:44.253750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd154bb2-f7ff-46eb-9ef9-76c1c5f16636 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.255551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd154bb2-f7ff-46eb-9ef9-76c1c5f16636 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.274695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.278654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 307/620] 10.222.0.78 () {56 vars in 1133 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:44.284762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154bb2-f7ff-46eb-9ef9-76c1c5f16636 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.329302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce488d-6e8f-4f57-b70a-91db68ba084b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.399297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9135202-672b-4fc7-a2be-75d94b8f3e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.399913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9135202-672b-4fc7-a2be-75d94b8f3e81 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.400192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9135202-672b-4fc7-a2be-75d94b8f3e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.400462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9135202-672b-4fc7-a2be-75d94b8f3e81 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.401123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9135202-672b-4fc7-a2be-75d94b8f3e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:44.401430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9135202-672b-4fc7-a2be-75d94b8f3e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.402816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9135202-672b-4fc7-a2be-75d94b8f3e81 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.431499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 314/621] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:44.436840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9135202-672b-4fc7-a2be-75d94b8f3e81 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.439193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154bb2-f7ff-46eb-9ef9-76c1c5f16636 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.476644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4036f5-60fb-4437-b7fb-af89957119ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.477346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4036f5-60fb-4437-b7fb-af89957119ae None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.477617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4036f5-60fb-4437-b7fb-af89957119ae None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.477930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4036f5-60fb-4437-b7fb-af89957119ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.549721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.553653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4036f5-60fb-4437-b7fb-af89957119ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.555094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4036f5-60fb-4437-b7fb-af89957119ae None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.615384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9135202-672b-4fc7-a2be-75d94b8f3e81 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.627340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.654039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505b9b6-1c7f-498c-ad2b-48a641facc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.654676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505b9b6-1c7f-498c-ad2b-48a641facc79 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.654955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505b9b6-1c7f-498c-ad2b-48a641facc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.655224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505b9b6-1c7f-498c-ad2b-48a641facc79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.660224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 308/622] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:44.661589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee31ac-ce6a-4c50-bc1d-68c8a8c5a634 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.662075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee31ac-ce6a-4c50-bc1d-68c8a8c5a634 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.662351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee31ac-ce6a-4c50-bc1d-68c8a8c5a634 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.663268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505b9b6-1c7f-498c-ad2b-48a641facc79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.663824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 309/623] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:44.664795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505b9b6-1c7f-498c-ad2b-48a641facc79 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.665360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee31ac-ce6a-4c50-bc1d-68c8a8c5a634 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-2505b9b6-1c7f-498c-ad2b-48a641facc79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:44.678624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:44.679802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:44.681055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 315/624] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:44.686309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f478ed7-fca8-4ddf-bf1f-e6fca617965a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.686581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f478ed7-fca8-4ddf-bf1f-e6fca617965a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.686965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f478ed7-fca8-4ddf-bf1f-e6fca617965a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.707883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9f478ed7-fca8-4ddf-bf1f-e6fca617965a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:44.709101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9f478ed7-fca8-4ddf-bf1f-e6fca617965a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:44.771570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.776290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee31ac-ce6a-4c50-bc1d-68c8a8c5a634 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.785983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 316/625] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:44.787175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f478ed7-fca8-4ddf-bf1f-e6fca617965a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.814673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde23199-c69d-4969-8399-0fc956387f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.815274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde23199-c69d-4969-8399-0fc956387f73 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.815552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde23199-c69d-4969-8399-0fc956387f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.815820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde23199-c69d-4969-8399-0fc956387f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.826933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde23199-c69d-4969-8399-0fc956387f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.828818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde23199-c69d-4969-8399-0fc956387f73 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-fde23199-c69d-4969-8399-0fc956387f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:44.844174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:44.845347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:44.846232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:44.846232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:44.846232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:44.846232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 310/626] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:44.858097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde23199-c69d-4969-8399-0fc956387f73 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.900337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.904279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f478ed7-fca8-4ddf-bf1f-e6fca617965a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.947175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097218c4-d039-481e-a652-e08707e40d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.948139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097218c4-d039-481e-a652-e08707e40d53 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.948629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097218c4-d039-481e-a652-e08707e40d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.949109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097218c4-d039-481e-a652-e08707e40d53 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.950035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097218c4-d039-481e-a652-e08707e40d53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:44.950421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097218c4-d039-481e-a652-e08707e40d53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.951943 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097218c4-d039-481e-a652-e08707e40d53 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.975262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 317/627] 10.222.0.78 () {56 vars in 1135 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:44.981023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af1721-95f1-4f62-9030-764078cca9b1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.981354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af1721-95f1-4f62-9030-764078cca9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.981629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af1721-95f1-4f62-9030-764078cca9b1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.983252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 318/628] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:44.985178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af1721-95f1-4f62-9030-764078cca9b1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.992193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde23199-c69d-4969-8399-0fc956387f73 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.037699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7c91f-29ae-4ea8-a0ca-99ef31b7296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.038508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7c91f-29ae-4ea8-a0ca-99ef31b7296b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.038986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7c91f-29ae-4ea8-a0ca-99ef31b7296b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.039334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7c91f-29ae-4ea8-a0ca-99ef31b7296b None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.040552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7c91f-29ae-4ea8-a0ca-99ef31b7296b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:45.040935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7c91f-29ae-4ea8-a0ca-99ef31b7296b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.042804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7c91f-29ae-4ea8-a0ca-99ef31b7296b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.060758 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 311/629] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:45.064094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ec54e-19d4-4577-985d-5cd1d7fedfbe None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.064396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ec54e-19d4-4577-985d-5cd1d7fedfbe None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.064648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ec54e-19d4-4577-985d-5cd1d7fedfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.087068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-733ec54e-19d4-4577-985d-5cd1d7fedfbe None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:45.088503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-733ec54e-19d4-4577-985d-5cd1d7fedfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:45.101467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.116512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af1721-95f1-4f62-9030-764078cca9b1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.159341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b973aa-5ffc-4049-8755-059c7e9b65b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.159954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b973aa-5ffc-4049-8755-059c7e9b65b8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.160235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b973aa-5ffc-4049-8755-059c7e9b65b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.160501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b973aa-5ffc-4049-8755-059c7e9b65b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.167765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 312/630] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:45.168605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ec54e-19d4-4577-985d-5cd1d7fedfbe None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.179777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b973aa-5ffc-4049-8755-059c7e9b65b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.181580 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b973aa-5ffc-4049-8755-059c7e9b65b8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-15b973aa-5ffc-4049-8755-059c7e9b65b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:45.196963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:45.197937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:45.199359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:45.199359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:45.199359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:45.199359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 319/631] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:45.199359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b973aa-5ffc-4049-8755-059c7e9b65b8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.212324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.286042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ec54e-19d4-4577-985d-5cd1d7fedfbe None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.310002 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b973aa-5ffc-4049-8755-059c7e9b65b8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.323944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d0733-08f2-4c4d-b3fa-bd0fdfee1e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.324553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d0733-08f2-4c4d-b3fa-bd0fdfee1e8f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.324828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d0733-08f2-4c4d-b3fa-bd0fdfee1e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.325110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d0733-08f2-4c4d-b3fa-bd0fdfee1e8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.331872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d0733-08f2-4c4d-b3fa-bd0fdfee1e8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.333326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d0733-08f2-4c4d-b3fa-bd0fdfee1e8f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.335783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-321d0733-08f2-4c4d-b3fa-bd0fdfee1e8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:45.347520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:45.348573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:45.349497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:45.349497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:45.349497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:45.349497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:45.349497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:45.349497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:45.349497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:45.349497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:45.349497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 313/632] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:45.350166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0c1ed-4dd6-48f8-8c8d-832f5dcedfe1 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.350418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0c1ed-4dd6-48f8-8c8d-832f5dcedfe1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.350681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0c1ed-4dd6-48f8-8c8d-832f5dcedfe1 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.352022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 314/633] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:45.352631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0c1ed-4dd6-48f8-8c8d-832f5dcedfe1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.361986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82c579-0ff4-4ecf-aba6-c2301094684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.362620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82c579-0ff4-4ecf-aba6-c2301094684c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.362898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82c579-0ff4-4ecf-aba6-c2301094684c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.363593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82c579-0ff4-4ecf-aba6-c2301094684c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.377408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82c579-0ff4-4ecf-aba6-c2301094684c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4d82c579-0ff4-4ecf-aba6-c2301094684c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:45.384731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:45.385667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:45.386507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:45.387155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 320/634] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:45.387808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82c579-0ff4-4ecf-aba6-c2301094684c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.464105 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0c1ed-4dd6-48f8-8c8d-832f5dcedfe1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.495505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82c579-0ff4-4ecf-aba6-c2301094684c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.498013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5816a6d-b1e6-4412-9877-3152bb452723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.498620 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5816a6d-b1e6-4412-9877-3152bb452723 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.498903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5816a6d-b1e6-4412-9877-3152bb452723 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.499178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5816a6d-b1e6-4412-9877-3152bb452723 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.499955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5816a6d-b1e6-4412-9877-3152bb452723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:45.500240 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5816a6d-b1e6-4412-9877-3152bb452723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.501815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5816a6d-b1e6-4412-9877-3152bb452723 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.513477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 315/635] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:45.514867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256616c-0402-48e4-9dce-030b02936cf6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.515155 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256616c-0402-48e4-9dce-030b02936cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.515507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256616c-0402-48e4-9dce-030b02936cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.532240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50953ab-b506-483a-8c4d-aac67d6403cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.532745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50953ab-b506-483a-8c4d-aac67d6403cb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.532943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8256616c-0402-48e4-9dce-030b02936cf6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:45.533047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50953ab-b506-483a-8c4d-aac67d6403cb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.533299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50953ab-b506-483a-8c4d-aac67d6403cb None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.533972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50953ab-b506-483a-8c4d-aac67d6403cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:45.534168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8256616c-0402-48e4-9dce-030b02936cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:45.534265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50953ab-b506-483a-8c4d-aac67d6403cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.535673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50953ab-b506-483a-8c4d-aac67d6403cb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.550065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 321/636] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:45.550899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50953ab-b506-483a-8c4d-aac67d6403cb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.594414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.605872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 316/637] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:45.606625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256616c-0402-48e4-9dce-030b02936cf6 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.663960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50953ab-b506-483a-8c4d-aac67d6403cb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.693963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.700539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f22c48-790c-4b1d-9544-b2879364d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.701669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22c48-790c-4b1d-9544-b2879364d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.702087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22c48-790c-4b1d-9544-b2879364d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.702496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22c48-790c-4b1d-9544-b2879364d4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.703861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f22c48-790c-4b1d-9544-b2879364d4d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:45.704317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f22c48-790c-4b1d-9544-b2879364d4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.705837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f22c48-790c-4b1d-9544-b2879364d4d9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.710441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256616c-0402-48e4-9dce-030b02936cf6 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.714504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 322/638] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:45.715429 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f22c48-790c-4b1d-9544-b2879364d4d9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.747304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6ed88-a049-473e-b502-608ac26bfe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.747935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6ed88-a049-473e-b502-608ac26bfe08 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.748228 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6ed88-a049-473e-b502-608ac26bfe08 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.748484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6ed88-a049-473e-b502-608ac26bfe08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.758840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6ed88-a049-473e-b502-608ac26bfe08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-70e6ed88-a049-473e-b502-608ac26bfe08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:45.763599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:45.764543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:45.765965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 317/639] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:45.766856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6ed88-a049-473e-b502-608ac26bfe08 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.825841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f22c48-790c-4b1d-9544-b2879364d4d9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.826526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.859729 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e04a5-886f-4ac7-b1bf-88870622d92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.860269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e04a5-886f-4ac7-b1bf-88870622d92b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.860544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e04a5-886f-4ac7-b1bf-88870622d92b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.860811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e04a5-886f-4ac7-b1bf-88870622d92b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.867706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e04a5-886f-4ac7-b1bf-88870622d92b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.869168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e04a5-886f-4ac7-b1bf-88870622d92b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-447e04a5-886f-4ac7-b1bf-88870622d92b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:45.880327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:45.881372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:45.882269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:45.882269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:45.882269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:45.882269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:45.882269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:45.882269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:45.882269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:45.882269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:45.882269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 323/640] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:45.882770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e04a5-886f-4ac7-b1bf-88870622d92b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.888135 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6ed88-a049-473e-b502-608ac26bfe08 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.927478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ff859-af32-4b21-8765-f687a7509277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.928075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ff859-af32-4b21-8765-f687a7509277 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.928357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ff859-af32-4b21-8765-f687a7509277 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.928621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ff859-af32-4b21-8765-f687a7509277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.929369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ff859-af32-4b21-8765-f687a7509277 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=f9b88c495fcb47a289a6a45276c891e0) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:45.929673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ff859-af32-4b21-8765-f687a7509277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.931101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ff859-af32-4b21-8765-f687a7509277 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.935016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.937238 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 318/641] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/role_assignments?user.id=f9b88c495fcb47a289a6a45276c891e0&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:45.937238 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ff859-af32-4b21-8765-f687a7509277 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.997350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e04a5-886f-4ac7-b1bf-88870622d92b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.037844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63008b9a-d8c3-42db-a7a6-1a965c906abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.038601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63008b9a-d8c3-42db-a7a6-1a965c906abb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.038905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63008b9a-d8c3-42db-a7a6-1a965c906abb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.039143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63008b9a-d8c3-42db-a7a6-1a965c906abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.051024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63008b9a-d8c3-42db-a7a6-1a965c906abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.052495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63008b9a-d8c3-42db-a7a6-1a965c906abb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.059493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ff859-af32-4b21-8765-f687a7509277 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-63008b9a-d8c3-42db-a7a6-1a965c906abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:46.077263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:46.078569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:46.079449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:46.079449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:46.079449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:46.079449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 324/642] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:46.079449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63008b9a-d8c3-42db-a7a6-1a965c906abb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.082209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.105939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283a283-f0ea-4d89-b421-f04721f1130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.106540 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283a283-f0ea-4d89-b421-f04721f1130b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.106844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283a283-f0ea-4d89-b421-f04721f1130b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.107118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283a283-f0ea-4d89-b421-f04721f1130b None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.107776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283a283-f0ea-4d89-b421-f04721f1130b None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:46.108074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283a283-f0ea-4d89-b421-f04721f1130b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.109517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283a283-f0ea-4d89-b421-f04721f1130b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.120132 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 319/643] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:46.120836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283a283-f0ea-4d89-b421-f04721f1130b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.192087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63008b9a-d8c3-42db-a7a6-1a965c906abb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.203740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.228197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283a283-f0ea-4d89-b421-f04721f1130b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.228551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb913d-5e8d-4df6-925e-beca82773c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.229189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb913d-5e8d-4df6-925e-beca82773c04 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.229468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb913d-5e8d-4df6-925e-beca82773c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.229742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb913d-5e8d-4df6-925e-beca82773c04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.240610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb913d-5e8d-4df6-925e-beca82773c04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-01eb913d-5e8d-4df6-925e-beca82773c04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:46.245898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:46.246840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:46.247703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:46.252337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 325/644] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:46.253234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb913d-5e8d-4df6-925e-beca82773c04 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.272025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a465b7a-d059-40ab-b8f0-f0a94fbd6361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.272727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a465b7a-d059-40ab-b8f0-f0a94fbd6361 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.273009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a465b7a-d059-40ab-b8f0-f0a94fbd6361 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.273308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a465b7a-d059-40ab-b8f0-f0a94fbd6361 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.274000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a465b7a-d059-40ab-b8f0-f0a94fbd6361 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:46.274301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a465b7a-d059-40ab-b8f0-f0a94fbd6361 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.275708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a465b7a-d059-40ab-b8f0-f0a94fbd6361 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.287964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 320/645] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:46.288644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a465b7a-d059-40ab-b8f0-f0a94fbd6361 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.325258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.358591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb913d-5e8d-4df6-925e-beca82773c04 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.394946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedda01-b65c-46e6-98da-c718b7ebdc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.395782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedda01-b65c-46e6-98da-c718b7ebdc0c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.396215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedda01-b65c-46e6-98da-c718b7ebdc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.396485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedda01-b65c-46e6-98da-c718b7ebdc0c None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.397292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedda01-b65c-46e6-98da-c718b7ebdc0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:46.397496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedda01-b65c-46e6-98da-c718b7ebdc0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.398840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedda01-b65c-46e6-98da-c718b7ebdc0c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.403533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a465b7a-d059-40ab-b8f0-f0a94fbd6361 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.418309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 326/646] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:46.418309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedda01-b65c-46e6-98da-c718b7ebdc0c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.443507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.446886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd4a61-54ab-4f3b-b3e7-72d0817247db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.447509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd4a61-54ab-4f3b-b3e7-72d0817247db None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.447789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd4a61-54ab-4f3b-b3e7-72d0817247db None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.448055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd4a61-54ab-4f3b-b3e7-72d0817247db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.460077 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd4a61-54ab-4f3b-b3e7-72d0817247db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.461684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd4a61-54ab-4f3b-b3e7-72d0817247db None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-eccd4a61-54ab-4f3b-b3e7-72d0817247db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:46.477819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:46.479009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:46.480185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:46.480185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:46.480185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:46.483940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 321/647] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:46.485539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd4a61-54ab-4f3b-b3e7-72d0817247db None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.527549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedda01-b65c-46e6-98da-c718b7ebdc0c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.563173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2a841-7fcb-4ad5-9da3-5cc6b9e0f724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.563810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2a841-7fcb-4ad5-9da3-5cc6b9e0f724 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.564103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2a841-7fcb-4ad5-9da3-5cc6b9e0f724 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.564367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2a841-7fcb-4ad5-9da3-5cc6b9e0f724 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.565031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2a841-7fcb-4ad5-9da3-5cc6b9e0f724 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:46.565370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2a841-7fcb-4ad5-9da3-5cc6b9e0f724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.566765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2a841-7fcb-4ad5-9da3-5cc6b9e0f724 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.575513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 327/648] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:46.576121 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2a841-7fcb-4ad5-9da3-5cc6b9e0f724 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.594707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.607353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd4a61-54ab-4f3b-b3e7-72d0817247db None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.645381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35982bf7-6c09-45ae-a753-961fa0e3e787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.645963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35982bf7-6c09-45ae-a753-961fa0e3e787 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.646236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35982bf7-6c09-45ae-a753-961fa0e3e787 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.646511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35982bf7-6c09-45ae-a753-961fa0e3e787 None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/c53b9bcd675940c19febbc8fefb1f28b/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.661805 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35982bf7-6c09-45ae-a753-961fa0e3e787 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=c53b9bcd675940c19febbc8fefb1f28b, role_id=fd7dbf5fe1ae4cdf8a10f67c6921d3da)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.663302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35982bf7-6c09-45ae-a753-961fa0e3e787 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.675993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.689308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 322/649] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:46 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/c53b9bcd675940c19febbc8fefb1f28b/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:46.689308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35982bf7-6c09-45ae-a753-961fa0e3e787 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.709481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2a841-7fcb-4ad5-9da3-5cc6b9e0f724 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.760741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11330f98-c845-4be6-90d9-53ce0808c250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.761687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11330f98-c845-4be6-90d9-53ce0808c250 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.762963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11330f98-c845-4be6-90d9-53ce0808c250 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.763246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11330f98-c845-4be6-90d9-53ce0808c250 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.773794 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11330f98-c845-4be6-90d9-53ce0808c250 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.775648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11330f98-c845-4be6-90d9-53ce0808c250 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-11330f98-c845-4be6-90d9-53ce0808c250 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:46.793057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:46.794358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 328/650] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:46.795374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11330f98-c845-4be6-90d9-53ce0808c250 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.804596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35982bf7-6c09-45ae-a753-961fa0e3e787 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.849069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c1fcc-5a1d-407f-afd7-59b354387e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.849474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c1fcc-5a1d-407f-afd7-59b354387e92 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.849747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c1fcc-5a1d-407f-afd7-59b354387e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.850013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c1fcc-5a1d-407f-afd7-59b354387e92 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.852701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c1fcc-5a1d-407f-afd7-59b354387e92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:46.852701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c1fcc-5a1d-407f-afd7-59b354387e92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.854448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c1fcc-5a1d-407f-afd7-59b354387e92 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.863657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.870795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 323/651] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:46.872571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c1fcc-5a1d-407f-afd7-59b354387e92 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.941575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11330f98-c845-4be6-90d9-53ce0808c250 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.979537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef28723-741a-4269-bf7c-1a6d65f0241b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.980166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef28723-741a-4269-bf7c-1a6d65f0241b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.980504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef28723-741a-4269-bf7c-1a6d65f0241b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.980823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef28723-741a-4269-bf7c-1a6d65f0241b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.994745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef28723-741a-4269-bf7c-1a6d65f0241b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.996456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef28723-741a-4269-bf7c-1a6d65f0241b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.004746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c1fcc-5a1d-407f-afd7-59b354387e92 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.005537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4ef28723-741a-4269-bf7c-1a6d65f0241b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:47.017566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:47.018582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:47.019462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:47.019462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:47.019462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:47.019462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 329/652] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:47.019708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef28723-741a-4269-bf7c-1a6d65f0241b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.059644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd914b6e-590e-4b45-a10b-45e2f1ac8f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.060861 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd914b6e-590e-4b45-a10b-45e2f1ac8f18 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.061646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd914b6e-590e-4b45-a10b-45e2f1ac8f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.062628 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd914b6e-590e-4b45-a10b-45e2f1ac8f18 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.064169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd914b6e-590e-4b45-a10b-45e2f1ac8f18 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:47.064695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd914b6e-590e-4b45-a10b-45e2f1ac8f18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.067152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd914b6e-590e-4b45-a10b-45e2f1ac8f18 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.091810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 324/653] 10.222.0.78 () {56 vars in 1137 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:47.099102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd914b6e-590e-4b45-a10b-45e2f1ac8f18 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.167981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef28723-741a-4269-bf7c-1a6d65f0241b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.179775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.216801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae8768-96fd-4041-8c42-f81fcdd708cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.217513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae8768-96fd-4041-8c42-f81fcdd708cd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.217706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae8768-96fd-4041-8c42-f81fcdd708cd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.218404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae8768-96fd-4041-8c42-f81fcdd708cd None admin] PATH_INFO: `/v3/users/glance` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.229332 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae8768-96fd-4041-8c42-f81fcdd708cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.231092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae8768-96fd-4041-8c42-f81fcdd708cd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b3ae8768-96fd-4041-8c42-f81fcdd708cd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:47.246509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:47.247471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:48:47.248418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:47.248949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd914b6e-590e-4b45-a10b-45e2f1ac8f18 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.251099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 330/654] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:47.252205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae8768-96fd-4041-8c42-f81fcdd708cd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.291674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9619-aa32-474d-907f-fdcb33f87dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.292354 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9619-aa32-474d-907f-fdcb33f87dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.292614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9619-aa32-474d-907f-fdcb33f87dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.292931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9619-aa32-474d-907f-fdcb33f87dc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.294254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee9619-aa32-474d-907f-fdcb33f87dc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:47.294254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee9619-aa32-474d-907f-fdcb33f87dc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.295253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee9619-aa32-474d-907f-fdcb33f87dc6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.307594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 325/655] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:47.313488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9619-aa32-474d-907f-fdcb33f87dc6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.319696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.366340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae8768-96fd-4041-8c42-f81fcdd708cd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.402200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bcfdb-62e0-47b4-8f35-454208bb39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.403187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bcfdb-62e0-47b4-8f35-454208bb39fe None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.403668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bcfdb-62e0-47b4-8f35-454208bb39fe None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.404147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bcfdb-62e0-47b4-8f35-454208bb39fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.424093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9619-aa32-474d-907f-fdcb33f87dc6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.442130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.462290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385be54-c886-438e-8387-a18d6a4324a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.462893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385be54-c886-438e-8387-a18d6a4324a0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.463174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385be54-c886-438e-8387-a18d6a4324a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.463445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385be54-c886-438e-8387-a18d6a4324a0 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.464174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385be54-c886-438e-8387-a18d6a4324a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:47.464480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385be54-c886-438e-8387-a18d6a4324a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.465937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385be54-c886-438e-8387-a18d6a4324a0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.478846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bcfdb-62e0-47b4-8f35-454208bb39fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.480555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bcfdb-62e0-47b4-8f35-454208bb39fe None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.485475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 326/656] 10.222.0.78 () {56 vars in 1137 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:47.486315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385be54-c886-438e-8387-a18d6a4324a0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.541623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.592452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 331/657] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:47.593119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385be54-c886-438e-8387-a18d6a4324a0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.594596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bcfdb-62e0-47b4-8f35-454208bb39fe None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.633109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620bf6-430b-44a3-87a0-6416aa4824ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.633766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620bf6-430b-44a3-87a0-6416aa4824ff None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.634049 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620bf6-430b-44a3-87a0-6416aa4824ff None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.634321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620bf6-430b-44a3-87a0-6416aa4824ff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.642763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3620bf6-430b-44a3-87a0-6416aa4824ff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.644220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3620bf6-430b-44a3-87a0-6416aa4824ff None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-e3620bf6-430b-44a3-87a0-6416aa4824ff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:47.657627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:47.658639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 327/658] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:47.659995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620bf6-430b-44a3-87a0-6416aa4824ff None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.702512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.707710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bcfdb-62e0-47b4-8f35-454208bb39fe None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.745359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07f7ff-0c05-425c-bce5-bcbe0e228453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.746313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07f7ff-0c05-425c-bce5-bcbe0e228453 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.746800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07f7ff-0c05-425c-bce5-bcbe0e228453 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.747268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07f7ff-0c05-425c-bce5-bcbe0e228453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.777197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620bf6-430b-44a3-87a0-6416aa4824ff None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.778370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.815072 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a752a5d-af64-4ad5-a2eb-99a731b6f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.815761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a752a5d-af64-4ad5-a2eb-99a731b6f696 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.816225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a752a5d-af64-4ad5-a2eb-99a731b6f696 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.816345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a752a5d-af64-4ad5-a2eb-99a731b6f696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.823456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07f7ff-0c05-425c-bce5-bcbe0e228453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.825212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07f7ff-0c05-425c-bce5-bcbe0e228453 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.829287 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a752a5d-af64-4ad5-a2eb-99a731b6f696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.830518 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a752a5d-af64-4ad5-a2eb-99a731b6f696 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3a752a5d-af64-4ad5-a2eb-99a731b6f696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:47.846662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:47.847661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:47.848567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:47.848567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:47.848567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:47.852470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 328/659] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:47.853421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a752a5d-af64-4ad5-a2eb-99a731b6f696 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.932915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 332/660] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:47.934235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07f7ff-0c05-425c-bce5-bcbe0e228453 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.949455 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.967630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a752a5d-af64-4ad5-a2eb-99a731b6f696 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.994735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.004370 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceae5b-cb20-49df-b9bd-7d3cd8040601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.004898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceae5b-cb20-49df-b9bd-7d3cd8040601 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.005258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceae5b-cb20-49df-b9bd-7d3cd8040601 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.005539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceae5b-cb20-49df-b9bd-7d3cd8040601 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.006267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceae5b-cb20-49df-b9bd-7d3cd8040601 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:48.006620 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceae5b-cb20-49df-b9bd-7d3cd8040601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.008765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceae5b-cb20-49df-b9bd-7d3cd8040601 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.026565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 329/661] 10.222.0.78 () {56 vars in 1139 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:48.032378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceae5b-cb20-49df-b9bd-7d3cd8040601 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.041593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07f7ff-0c05-425c-bce5-bcbe0e228453 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.079083 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c960874-a37b-4166-8f16-169b50e09c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.079685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c960874-a37b-4166-8f16-169b50e09c2c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.079958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c960874-a37b-4166-8f16-169b50e09c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.080225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c960874-a37b-4166-8f16-169b50e09c2c None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.080855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c960874-a37b-4166-8f16-169b50e09c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:48.081182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c960874-a37b-4166-8f16-169b50e09c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.082562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c960874-a37b-4166-8f16-169b50e09c2c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.095258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 333/662] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:48.096520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb9ec3-20d3-48f6-9a98-2ea8b1b53074 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.096810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb9ec3-20d3-48f6-9a98-2ea8b1b53074 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.097109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb9ec3-20d3-48f6-9a98-2ea8b1b53074 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.098450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 334/663] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:48.099027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb9ec3-20d3-48f6-9a98-2ea8b1b53074 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.146514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceae5b-cb20-49df-b9bd-7d3cd8040601 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.183577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc7fd9-0170-46e2-bc5c-a8538fd1c594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.184201 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc7fd9-0170-46e2-bc5c-a8538fd1c594 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.184484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc7fd9-0170-46e2-bc5c-a8538fd1c594 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.184753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc7fd9-0170-46e2-bc5c-a8538fd1c594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.196870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc7fd9-0170-46e2-bc5c-a8538fd1c594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.198340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc7fd9-0170-46e2-bc5c-a8538fd1c594 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.200869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.206115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb9ec3-20d3-48f6-9a98-2ea8b1b53074 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-2bcc7fd9-0170-46e2-bc5c-a8538fd1c594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:48.216516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:48.217640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:48.218553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:48.218553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:48.218553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:48.222345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 330/664] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:48.223190 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc7fd9-0170-46e2-bc5c-a8538fd1c594 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.227460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.252693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8c50-7110-481e-92ce-485b822f7718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.253660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8c50-7110-481e-92ce-485b822f7718 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.253936 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8c50-7110-481e-92ce-485b822f7718 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.254201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8c50-7110-481e-92ce-485b822f7718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.327242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8c50-7110-481e-92ce-485b822f7718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.329647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8c50-7110-481e-92ce-485b822f7718 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.337776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc7fd9-0170-46e2-bc5c-a8538fd1c594 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.372150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf48862-c08e-4c78-bd56-34484a628f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.372776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf48862-c08e-4c78-bd56-34484a628f46 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.373060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf48862-c08e-4c78-bd56-34484a628f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.373358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf48862-c08e-4c78-bd56-34484a628f46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.381695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf48862-c08e-4c78-bd56-34484a628f46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.383145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf48862-c08e-4c78-bd56-34484a628f46 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7cf48862-c08e-4c78-bd56-34484a628f46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:48.395863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:48.396819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:48.397754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:48.397754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:48.397754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:48.397754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:48.397754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:48.397754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:48.397754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:48.397754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:48.397754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 331/665] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:48.398258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8965a-1ebd-498e-86cb-ad72232ffcad None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.398633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8965a-1ebd-498e-86cb-ad72232ffcad None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.398633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8965a-1ebd-498e-86cb-ad72232ffcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.418016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-14a8965a-1ebd-498e-86cb-ad72232ffcad None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:48.419613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-14a8965a-1ebd-498e-86cb-ad72232ffcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:48.444932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 335/666] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:48.445755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8c50-7110-481e-92ce-485b822f7718 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.464470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.481441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.505203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 332/667] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:48.505203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8965a-1ebd-498e-86cb-ad72232ffcad None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.557165 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8c50-7110-481e-92ce-485b822f7718 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.597674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f749f-0d2e-4894-adeb-5fdb979da0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.598928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f749f-0d2e-4894-adeb-5fdb979da0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.599316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f749f-0d2e-4894-adeb-5fdb979da0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.599712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f749f-0d2e-4894-adeb-5fdb979da0c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.600575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f749f-0d2e-4894-adeb-5fdb979da0c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:48.600982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f749f-0d2e-4894-adeb-5fdb979da0c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.602610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f749f-0d2e-4894-adeb-5fdb979da0c4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.615658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 336/668] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:48.616732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f749f-0d2e-4894-adeb-5fdb979da0c4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.639720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8965a-1ebd-498e-86cb-ad72232ffcad None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.690587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3724e3b-428a-4988-ad1f-175385ec7508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.691445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3724e3b-428a-4988-ad1f-175385ec7508 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.692161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3724e3b-428a-4988-ad1f-175385ec7508 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.692600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3724e3b-428a-4988-ad1f-175385ec7508 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.694091 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3724e3b-428a-4988-ad1f-175385ec7508 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:48.694811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3724e3b-428a-4988-ad1f-175385ec7508 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.696978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3724e3b-428a-4988-ad1f-175385ec7508 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.708953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.716225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 333/669] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:48.717067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3724e3b-428a-4988-ad1f-175385ec7508 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.733170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f749f-0d2e-4894-adeb-5fdb979da0c4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.768306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a329f-232a-4ccc-bf04-63d67bc1f72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.768929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a329f-232a-4ccc-bf04-63d67bc1f72d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.769240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a329f-232a-4ccc-bf04-63d67bc1f72d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.769371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.769519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a329f-232a-4ccc-bf04-63d67bc1f72d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.780737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a329f-232a-4ccc-bf04-63d67bc1f72d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.782857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a329f-232a-4ccc-bf04-63d67bc1f72d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-504a329f-232a-4ccc-bf04-63d67bc1f72d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:48.797284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:48.798256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:48.799125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:48.799125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:48.799125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:48.799125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 337/670] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:48.799354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a329f-232a-4ccc-bf04-63d67bc1f72d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.830766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3724e3b-428a-4988-ad1f-175385ec7508 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.863928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841844af-baf3-434a-9475-80c38d25fc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.864560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841844af-baf3-434a-9475-80c38d25fc56 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.864847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841844af-baf3-434a-9475-80c38d25fc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.865135 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841844af-baf3-434a-9475-80c38d25fc56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.871827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841844af-baf3-434a-9475-80c38d25fc56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.873289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841844af-baf3-434a-9475-80c38d25fc56 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-841844af-baf3-434a-9475-80c38d25fc56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:48.885094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:48.885965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:48.886848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:48.886848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:48.886848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:48.886848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:48.886848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:48.886848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:48.886848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:48.886848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:48.886848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 334/671] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:48.887317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f663e-75c1-47a0-9c0b-6baac0da1a30 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.887629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f663e-75c1-47a0-9c0b-6baac0da1a30 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.887935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f663e-75c1-47a0-9c0b-6baac0da1a30 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.889062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 335/672] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:48.889771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f663e-75c1-47a0-9c0b-6baac0da1a30 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.913345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a329f-232a-4ccc-bf04-63d67bc1f72d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.943880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.950499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daad333-0e7f-466d-8aee-e13d41db52a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.951135 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad333-0e7f-466d-8aee-e13d41db52a2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.951418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad333-0e7f-466d-8aee-e13d41db52a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.951693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad333-0e7f-466d-8aee-e13d41db52a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.952414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daad333-0e7f-466d-8aee-e13d41db52a2 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=97e64ee3c8ee48c2ad8f351fb2a78bc8) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:48.952720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daad333-0e7f-466d-8aee-e13d41db52a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.954917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daad333-0e7f-466d-8aee-e13d41db52a2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.960572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 338/673] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/role_assignments?user.id=97e64ee3c8ee48c2ad8f351fb2a78bc8&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:48.961372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daad333-0e7f-466d-8aee-e13d41db52a2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.998505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f663e-75c1-47a0-9c0b-6baac0da1a30 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.007641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.039564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1563b-e8e0-4bf1-bbd2-ffa58d27d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.040185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1563b-e8e0-4bf1-bbd2-ffa58d27d99a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.040464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1563b-e8e0-4bf1-bbd2-ffa58d27d99a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.040734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1563b-e8e0-4bf1-bbd2-ffa58d27d99a None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.041425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1563b-e8e0-4bf1-bbd2-ffa58d27d99a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:49.041728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1563b-e8e0-4bf1-bbd2-ffa58d27d99a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:49.043099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1563b-e8e0-4bf1-bbd2-ffa58d27d99a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:49.057722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 336/674] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:49.058408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1563b-e8e0-4bf1-bbd2-ffa58d27d99a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.111648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daad333-0e7f-466d-8aee-e13d41db52a2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.150367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d34dd-2098-4b48-9d9d-e3ece33e467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.151109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d34dd-2098-4b48-9d9d-e3ece33e467f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.151384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d34dd-2098-4b48-9d9d-e3ece33e467f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.151674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d34dd-2098-4b48-9d9d-e3ece33e467f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.164440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d34dd-2098-4b48-9d9d-e3ece33e467f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b70d34dd-2098-4b48-9d9d-e3ece33e467f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:49.169622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:49.170730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:49.172483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 339/675] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:49.173555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a836918-6024-46c6-974e-9b4d87afcdcc None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.173555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a836918-6024-46c6-974e-9b4d87afcdcc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.173555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a836918-6024-46c6-974e-9b4d87afcdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.180919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1563b-e8e0-4bf1-bbd2-ffa58d27d99a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.190884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8a836918-6024-46c6-974e-9b4d87afcdcc None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:49.192070 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8a836918-6024-46c6-974e-9b4d87afcdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:49.220576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e680b-46df-4777-8301-82a02d7e186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.221448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e680b-46df-4777-8301-82a02d7e186a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.222081 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e680b-46df-4777-8301-82a02d7e186a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.222501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e680b-46df-4777-8301-82a02d7e186a None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.223412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e680b-46df-4777-8301-82a02d7e186a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:49.223861 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e680b-46df-4777-8301-82a02d7e186a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:49.225446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e680b-46df-4777-8301-82a02d7e186a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:49.239512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 337/676] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:49.240547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e680b-46df-4777-8301-82a02d7e186a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.251879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.275861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 340/677] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:49.277699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.279229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a836918-6024-46c6-974e-9b4d87afcdcc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.377446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e680b-46df-4777-8301-82a02d7e186a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.411117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a836918-6024-46c6-974e-9b4d87afcdcc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.420306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7f14f-0175-489f-a4f4-b43e83243049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.421046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7f14f-0175-489f-a4f4-b43e83243049 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.421461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7f14f-0175-489f-a4f4-b43e83243049 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.423036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7f14f-0175-489f-a4f4-b43e83243049 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.432245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7f14f-0175-489f-a4f4-b43e83243049 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:49.434442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7f14f-0175-489f-a4f4-b43e83243049 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-0ce7f14f-0175-489f-a4f4-b43e83243049 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:49.450327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:49.451263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:49.452137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:49.452137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:49.452137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:49.452137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:49.452137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:49.452137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:49.452137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:49.452137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:49.455989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 338/678] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:49.456135 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0c2e9-0f51-4649-8d1a-e1cb3d4bc57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.456894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7f14f-0175-489f-a4f4-b43e83243049 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.457022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0c2e9-0f51-4649-8d1a-e1cb3d4bc57b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.457539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0c2e9-0f51-4649-8d1a-e1cb3d4bc57b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.457949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0c2e9-0f51-4649-8d1a-e1cb3d4bc57b None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.458747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0c2e9-0f51-4649-8d1a-e1cb3d4bc57b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:49.459187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0c2e9-0f51-4649-8d1a-e1cb3d4bc57b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:49.460715 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0c2e9-0f51-4649-8d1a-e1cb3d4bc57b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:49.472507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 341/679] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:49.473203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0c2e9-0f51-4649-8d1a-e1cb3d4bc57b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.525030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.575073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.585206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0c2e9-0f51-4649-8d1a-e1cb3d4bc57b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.591705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7f14f-0175-489f-a4f4-b43e83243049 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.622259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd9536-6220-40b5-8faa-445d5a5f1679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.622859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd9536-6220-40b5-8faa-445d5a5f1679 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.623134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd9536-6220-40b5-8faa-445d5a5f1679 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.623392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd9536-6220-40b5-8faa-445d5a5f1679 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.624259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd9536-6220-40b5-8faa-445d5a5f1679 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:49.624564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd9536-6220-40b5-8faa-445d5a5f1679 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:49.626334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd9536-6220-40b5-8faa-445d5a5f1679 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:49.627035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7030db-c3ae-400c-9c14-9999e6e67eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.627688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7030db-c3ae-400c-9c14-9999e6e67eaf None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.627917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7030db-c3ae-400c-9c14-9999e6e67eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.628193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7030db-c3ae-400c-9c14-9999e6e67eaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.628918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7030db-c3ae-400c-9c14-9999e6e67eaf None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=77a47e77cc1942618a3584862042c5f7) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:49.629261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7030db-c3ae-400c-9c14-9999e6e67eaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:49.630663 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7030db-c3ae-400c-9c14-9999e6e67eaf None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:49.636429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 339/680] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/role_assignments?user.id=77a47e77cc1942618a3584862042c5f7&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:49.637243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7030db-c3ae-400c-9c14-9999e6e67eaf None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.649801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 342/681] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:49.651142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd9536-6220-40b5-8faa-445d5a5f1679 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.767211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd9536-6220-40b5-8faa-445d5a5f1679 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.777204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.778458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7030db-c3ae-400c-9c14-9999e6e67eaf None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.820835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda984c5-6f3c-4858-b233-f70e365aea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.821521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda984c5-6f3c-4858-b233-f70e365aea6c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.821802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda984c5-6f3c-4858-b233-f70e365aea6c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.822073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda984c5-6f3c-4858-b233-f70e365aea6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.826505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482f52b-3dc8-4eed-9b3b-3fd7d7e23aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.827395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482f52b-3dc8-4eed-9b3b-3fd7d7e23aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.827677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482f52b-3dc8-4eed-9b3b-3fd7d7e23aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.828079 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482f52b-3dc8-4eed-9b3b-3fd7d7e23aa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.832848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda984c5-6f3c-4858-b233-f70e365aea6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-bda984c5-6f3c-4858-b233-f70e365aea6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:49.837826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:49.838730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:49.839625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:49.844167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482f52b-3dc8-4eed-9b3b-3fd7d7e23aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:49.844357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 343/682] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:49.845177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda984c5-6f3c-4858-b233-f70e365aea6c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.845992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482f52b-3dc8-4eed-9b3b-3fd7d7e23aa8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-6482f52b-3dc8-4eed-9b3b-3fd7d7e23aa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:49.864807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:49.865846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:49.866705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:49.866705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:49.866705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:49.866705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 340/683] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:49.866953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482f52b-3dc8-4eed-9b3b-3fd7d7e23aa8 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.870088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.983179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda984c5-6f3c-4858-b233-f70e365aea6c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.994886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482f52b-3dc8-4eed-9b3b-3fd7d7e23aa8 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.023243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be8481-18fe-4d8d-9def-9ca49da64321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.023849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be8481-18fe-4d8d-9def-9ca49da64321 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.024125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be8481-18fe-4d8d-9def-9ca49da64321 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.024389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be8481-18fe-4d8d-9def-9ca49da64321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.025123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be8481-18fe-4d8d-9def-9ca49da64321 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=1a5d841380c243209f55314b545ff56f) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:50.025430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be8481-18fe-4d8d-9def-9ca49da64321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.027013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be8481-18fe-4d8d-9def-9ca49da64321 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.030228 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a6869-667e-4808-b163-1a9d978b2cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.030832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a6869-667e-4808-b163-1a9d978b2cdb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.031654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a6869-667e-4808-b163-1a9d978b2cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.031922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a6869-667e-4808-b163-1a9d978b2cdb None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.032682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a6869-667e-4808-b163-1a9d978b2cdb None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:50.032984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a6869-667e-4808-b163-1a9d978b2cdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.034375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a6869-667e-4808-b163-1a9d978b2cdb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.034485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 344/684] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/role_assignments?user.id=1a5d841380c243209f55314b545ff56f&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:50.035477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be8481-18fe-4d8d-9def-9ca49da64321 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.043648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 341/685] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:50.054426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a6869-667e-4808-b163-1a9d978b2cdb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.075309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.160594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be8481-18fe-4d8d-9def-9ca49da64321 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.198979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6825a9-3a72-45ec-9218-6766b6ebe4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.199752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6825a9-3a72-45ec-9218-6766b6ebe4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.200036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6825a9-3a72-45ec-9218-6766b6ebe4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.200304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6825a9-3a72-45ec-9218-6766b6ebe4d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.200947 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6825a9-3a72-45ec-9218-6766b6ebe4d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:50.201267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6825a9-3a72-45ec-9218-6766b6ebe4d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.202641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6825a9-3a72-45ec-9218-6766b6ebe4d4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.211286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.217773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 345/686] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:50.224613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6825a9-3a72-45ec-9218-6766b6ebe4d4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.283660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a6869-667e-4808-b163-1a9d978b2cdb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.322674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898045f-e9d1-480c-a742-a793787b6226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.323304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898045f-e9d1-480c-a742-a793787b6226 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.323584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898045f-e9d1-480c-a742-a793787b6226 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.323855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898045f-e9d1-480c-a742-a793787b6226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.324193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.335693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6825a9-3a72-45ec-9218-6766b6ebe4d4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.338710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898045f-e9d1-480c-a742-a793787b6226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.340180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898045f-e9d1-480c-a742-a793787b6226 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c898045f-e9d1-480c-a742-a793787b6226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:50.356884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:50.357878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:50.358739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:50.358739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:50.358739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:50.358739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 342/687] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:50.364185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898045f-e9d1-480c-a742-a793787b6226 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.373493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697d72d-98e5-49e6-aab5-00e39342a0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.374124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697d72d-98e5-49e6-aab5-00e39342a0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.374401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697d72d-98e5-49e6-aab5-00e39342a0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.374712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697d72d-98e5-49e6-aab5-00e39342a0c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.384009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697d72d-98e5-49e6-aab5-00e39342a0c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.385620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697d72d-98e5-49e6-aab5-00e39342a0c8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3697d72d-98e5-49e6-aab5-00e39342a0c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:50.399342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:50.401583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:50.403771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 346/688] 10.222.0.78 () {56 vars in 1093 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:50.413097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697d72d-98e5-49e6-aab5-00e39342a0c8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.491334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898045f-e9d1-480c-a742-a793787b6226 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.531887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.532508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697d72d-98e5-49e6-aab5-00e39342a0c8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.573557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb18139-3dee-40ad-8ee2-ff5530e2bab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.574167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb18139-3dee-40ad-8ee2-ff5530e2bab9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.574444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb18139-3dee-40ad-8ee2-ff5530e2bab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.574716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb18139-3dee-40ad-8ee2-ff5530e2bab9 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.575438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb18139-3dee-40ad-8ee2-ff5530e2bab9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:50.575740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb18139-3dee-40ad-8ee2-ff5530e2bab9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.577263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb18139-3dee-40ad-8ee2-ff5530e2bab9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.581164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1d349-0b1c-48c2-90f2-6fb6153b1a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.581775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1d349-0b1c-48c2-90f2-6fb6153b1a7d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.582051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1d349-0b1c-48c2-90f2-6fb6153b1a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.582318 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1d349-0b1c-48c2-90f2-6fb6153b1a7d None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.582960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1d349-0b1c-48c2-90f2-6fb6153b1a7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:50.583259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1d349-0b1c-48c2-90f2-6fb6153b1a7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.584599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1d349-0b1c-48c2-90f2-6fb6153b1a7d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.597428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 347/689] 10.222.0.78 () {56 vars in 1133 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:50.607960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb18139-3dee-40ad-8ee2-ff5530e2bab9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.612689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 343/690] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:50.617872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1d349-0b1c-48c2-90f2-6fb6153b1a7d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.620681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.730422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb18139-3dee-40ad-8ee2-ff5530e2bab9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.780385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4befcd-5a9b-4c9a-b951-2d133808ed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.780385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4befcd-5a9b-4c9a-b951-2d133808ed63 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.780385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4befcd-5a9b-4c9a-b951-2d133808ed63 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.780385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4befcd-5a9b-4c9a-b951-2d133808ed63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.814600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1d349-0b1c-48c2-90f2-6fb6153b1a7d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.860014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662868e-61af-47a8-be44-f53bbccb4d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.860656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662868e-61af-47a8-be44-f53bbccb4d30 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.860938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662868e-61af-47a8-be44-f53bbccb4d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.861235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662868e-61af-47a8-be44-f53bbccb4d30 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.871674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662868e-61af-47a8-be44-f53bbccb4d30 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.873995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662868e-61af-47a8-be44-f53bbccb4d30 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9662868e-61af-47a8-be44-f53bbccb4d30 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:50.896189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:50.897293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:48:50.898197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:50.901697 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 344/691] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:50.907173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662868e-61af-47a8-be44-f53bbccb4d30 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.918139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4befcd-5a9b-4c9a-b951-2d133808ed63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.919598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4befcd-5a9b-4c9a-b951-2d133808ed63 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.941637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.965430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.063803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662868e-61af-47a8-be44-f53bbccb4d30 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.110433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 348/692] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:51.111709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d497df-1563-42ce-b7ad-da1bb2e127d0 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.111998 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d497df-1563-42ce-b7ad-da1bb2e127d0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.112269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d497df-1563-42ce-b7ad-da1bb2e127d0 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.113535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 349/693] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:51.122393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d497df-1563-42ce-b7ad-da1bb2e127d0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.123661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f80a5e-b3a5-4e31-a8ec-b10f0b8a1eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.124548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f80a5e-b3a5-4e31-a8ec-b10f0b8a1eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.129551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f80a5e-b3a5-4e31-a8ec-b10f0b8a1eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.130095 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f80a5e-b3a5-4e31-a8ec-b10f0b8a1eb0 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.131216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f80a5e-b3a5-4e31-a8ec-b10f0b8a1eb0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:51.131749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f80a5e-b3a5-4e31-a8ec-b10f0b8a1eb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.133572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f80a5e-b3a5-4e31-a8ec-b10f0b8a1eb0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:51.153117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 345/694] 10.222.0.78 () {56 vars in 1135 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:51.154476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a14bef-7e1e-4a4d-a98c-8bc31729f25c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.154831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a14bef-7e1e-4a4d-a98c-8bc31729f25c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.155112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a14bef-7e1e-4a4d-a98c-8bc31729f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.181222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c7a14bef-7e1e-4a4d-a98c-8bc31729f25c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:51.182851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c7a14bef-7e1e-4a4d-a98c-8bc31729f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:51.223956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d497df-1563-42ce-b7ad-da1bb2e127d0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.233599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.263240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d31879-bd2f-4dd7-9b1b-7e56b3d7baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.263864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d31879-bd2f-4dd7-9b1b-7e56b3d7baac None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.264136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d31879-bd2f-4dd7-9b1b-7e56b3d7baac None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.264399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d31879-bd2f-4dd7-9b1b-7e56b3d7baac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.275245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 346/695] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:51.276565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a14bef-7e1e-4a4d-a98c-8bc31729f25c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.289982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d31879-bd2f-4dd7-9b1b-7e56b3d7baac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.291433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d31879-bd2f-4dd7-9b1b-7e56b3d7baac None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:51.309446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-32d31879-bd2f-4dd7-9b1b-7e56b3d7baac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:51.322642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:51.323633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:51.325453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:51.325453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:51.325453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:51.325453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 350/696] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:51.334028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d31879-bd2f-4dd7-9b1b-7e56b3d7baac None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.389817 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a14bef-7e1e-4a4d-a98c-8bc31729f25c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.426828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296e25d-42c1-491c-b92e-a11b12aa24e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.427819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296e25d-42c1-491c-b92e-a11b12aa24e3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.428346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296e25d-42c1-491c-b92e-a11b12aa24e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.428820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296e25d-42c1-491c-b92e-a11b12aa24e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.490300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d31879-bd2f-4dd7-9b1b-7e56b3d7baac None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.506797 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296e25d-42c1-491c-b92e-a11b12aa24e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.509260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296e25d-42c1-491c-b92e-a11b12aa24e3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:51.526721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d61c8-e449-4e0e-810e-0e2755d45f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.527331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d61c8-e449-4e0e-810e-0e2755d45f7b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.527605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d61c8-e449-4e0e-810e-0e2755d45f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.527870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d61c8-e449-4e0e-810e-0e2755d45f7b None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.528531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d61c8-e449-4e0e-810e-0e2755d45f7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:51.528809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d61c8-e449-4e0e-810e-0e2755d45f7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.530193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d61c8-e449-4e0e-810e-0e2755d45f7b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:51.541475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.542825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 351/697] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:51.550963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d61c8-e449-4e0e-810e-0e2755d45f7b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.582245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.636538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 347/698] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:51.639025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296e25d-42c1-491c-b92e-a11b12aa24e3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.687312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d61c8-e449-4e0e-810e-0e2755d45f7b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.764076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a357f-c9e8-48e6-bda5-a4f7faca235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.764689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296e25d-42c1-491c-b92e-a11b12aa24e3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.764876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a357f-c9e8-48e6-bda5-a4f7faca235b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.764963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a357f-c9e8-48e6-bda5-a4f7faca235b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.765772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a357f-c9e8-48e6-bda5-a4f7faca235b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.777545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a357f-c9e8-48e6-bda5-a4f7faca235b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.779365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a357f-c9e8-48e6-bda5-a4f7faca235b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e67a357f-c9e8-48e6-bda5-a4f7faca235b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:51.795865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:51.797147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:51.798021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:51.798021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:51.798021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:51.798021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:51.798021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:51.798021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:51.798021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:51.798021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:51.799045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 352/699] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:51.801043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a357f-c9e8-48e6-bda5-a4f7faca235b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.809457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.815025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d268963-987f-4263-b82c-82db6f8f2d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.815804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d268963-987f-4263-b82c-82db6f8f2d4a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.816192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d268963-987f-4263-b82c-82db6f8f2d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.816571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d268963-987f-4263-b82c-82db6f8f2d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.831578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d268963-987f-4263-b82c-82db6f8f2d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.834264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d268963-987f-4263-b82c-82db6f8f2d4a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7d268963-987f-4263-b82c-82db6f8f2d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:51.851719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:51.852775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:51.853682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:51.853682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:51.853682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:51.858007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 348/700] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:51.859480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d268963-987f-4263-b82c-82db6f8f2d4a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.915558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.945538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a357f-c9e8-48e6-bda5-a4f7faca235b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.976809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d268963-987f-4263-b82c-82db6f8f2d4a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.994702 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca32ad9-3af7-48a2-bfe1-4179a9c145c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.995640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca32ad9-3af7-48a2-bfe1-4179a9c145c1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.996082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca32ad9-3af7-48a2-bfe1-4179a9c145c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.996353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca32ad9-3af7-48a2-bfe1-4179a9c145c1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.996938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca32ad9-3af7-48a2-bfe1-4179a9c145c1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.998335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca32ad9-3af7-48a2-bfe1-4179a9c145c1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-6ca32ad9-3af7-48a2-bfe1-4179a9c145c1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:52.010627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:52.011774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:48:52.013209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:52.017354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 353/701] 10.222.0.78 () {56 vars in 1129 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:52.018500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca32ad9-3af7-48a2-bfe1-4179a9c145c1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.024344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf51d83-b1ee-4d35-9dff-4a6a15780143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.024954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf51d83-b1ee-4d35-9dff-4a6a15780143 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.025283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf51d83-b1ee-4d35-9dff-4a6a15780143 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.025558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf51d83-b1ee-4d35-9dff-4a6a15780143 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.026271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf51d83-b1ee-4d35-9dff-4a6a15780143 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:52.026575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf51d83-b1ee-4d35-9dff-4a6a15780143 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.028017 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf51d83-b1ee-4d35-9dff-4a6a15780143 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.045136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 349/702] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:52.045136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b1032-1b0e-4755-8129-b70b2579b2ab None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.045136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b1032-1b0e-4755-8129-b70b2579b2ab None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.045136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b1032-1b0e-4755-8129-b70b2579b2ab None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.046178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 350/703] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:52.046984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b1032-1b0e-4755-8129-b70b2579b2ab None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.092802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.141176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca32ad9-3af7-48a2-bfe1-4179a9c145c1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.184016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b1032-1b0e-4755-8129-b70b2579b2ab None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.185229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67126b-9646-49b0-9e3d-c0d66e4f32be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.185846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67126b-9646-49b0-9e3d-c0d66e4f32be None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.186128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67126b-9646-49b0-9e3d-c0d66e4f32be None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.188050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67126b-9646-49b0-9e3d-c0d66e4f32be None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.189874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67126b-9646-49b0-9e3d-c0d66e4f32be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:52.190515 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67126b-9646-49b0-9e3d-c0d66e4f32be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.192001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67126b-9646-49b0-9e3d-c0d66e4f32be None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 354/704] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:52.210006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67126b-9646-49b0-9e3d-c0d66e4f32be None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.221879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.243482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38d83a-3f30-43f6-89f0-412893d31ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.244089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38d83a-3f30-43f6-89f0-412893d31ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.244369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38d83a-3f30-43f6-89f0-412893d31ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.244639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38d83a-3f30-43f6-89f0-412893d31ad1 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.245300 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38d83a-3f30-43f6-89f0-412893d31ad1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:52.245778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38d83a-3f30-43f6-89f0-412893d31ad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.247111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38d83a-3f30-43f6-89f0-412893d31ad1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.254061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 351/705] 10.222.0.78 () {56 vars in 1124 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:52.257677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3926221-38ca-41e9-8618-844ca6c789d7 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.257677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3926221-38ca-41e9-8618-844ca6c789d7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.258749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3926221-38ca-41e9-8618-844ca6c789d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.279693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b3926221-38ca-41e9-8618-844ca6c789d7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:52.280993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b3926221-38ca-41e9-8618-844ca6c789d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:52.345463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67126b-9646-49b0-9e3d-c0d66e4f32be None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 352/706] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:52.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3926221-38ca-41e9-8618-844ca6c789d7 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.385447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120a1dd-9233-452d-819c-8b19222b53b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.386046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120a1dd-9233-452d-819c-8b19222b53b3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.386318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120a1dd-9233-452d-819c-8b19222b53b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.386600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120a1dd-9233-452d-819c-8b19222b53b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.387294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120a1dd-9233-452d-819c-8b19222b53b3 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=f9b88c495fcb47a289a6a45276c891e0) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:52.387599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120a1dd-9233-452d-819c-8b19222b53b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.388777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.390182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120a1dd-9233-452d-819c-8b19222b53b3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.401657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 355/707] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/role_assignments?user.id=f9b88c495fcb47a289a6a45276c891e0&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:52.402655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120a1dd-9233-452d-819c-8b19222b53b3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.471677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3926221-38ca-41e9-8618-844ca6c789d7 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.510208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbc2ed-59f3-4918-b3be-19ac4d7bba4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.512110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbc2ed-59f3-4918-b3be-19ac4d7bba4f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.512597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbc2ed-59f3-4918-b3be-19ac4d7bba4f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.513062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbc2ed-59f3-4918-b3be-19ac4d7bba4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.522868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dbc2ed-59f3-4918-b3be-19ac4d7bba4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.524778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dbc2ed-59f3-4918-b3be-19ac4d7bba4f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.536014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-78dbc2ed-59f3-4918-b3be-19ac4d7bba4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:52.540705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:52.541667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:52.542974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:52.542974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:52.542974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:52.542974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:52.542974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:52.542974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:52.542974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:52.542974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:52.545219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 353/708] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:52.551391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbc2ed-59f3-4918-b3be-19ac4d7bba4f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.607274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120a1dd-9233-452d-819c-8b19222b53b3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.655151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.667739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e11db-79ae-4ccc-a3f1-e8f5cc02e042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.668365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e11db-79ae-4ccc-a3f1-e8f5cc02e042 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.668643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e11db-79ae-4ccc-a3f1-e8f5cc02e042 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.668914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e11db-79ae-4ccc-a3f1-e8f5cc02e042 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.669602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e11db-79ae-4ccc-a3f1-e8f5cc02e042 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:52.671815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbc2ed-59f3-4918-b3be-19ac4d7bba4f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.671929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e11db-79ae-4ccc-a3f1-e8f5cc02e042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.671929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e11db-79ae-4ccc-a3f1-e8f5cc02e042 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.678665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 356/709] 10.222.0.78 () {56 vars in 1124 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:52.685147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e11db-79ae-4ccc-a3f1-e8f5cc02e042 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.725882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffc9a6-b636-49fc-86a2-cd41c201a747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.726422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffc9a6-b636-49fc-86a2-cd41c201a747 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.726696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffc9a6-b636-49fc-86a2-cd41c201a747 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.726967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffc9a6-b636-49fc-86a2-cd41c201a747 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.727682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffc9a6-b636-49fc-86a2-cd41c201a747 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:52.729433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffc9a6-b636-49fc-86a2-cd41c201a747 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.731153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffc9a6-b636-49fc-86a2-cd41c201a747 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.757235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 354/710] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:52.759552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffc9a6-b636-49fc-86a2-cd41c201a747 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.824938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e11db-79ae-4ccc-a3f1-e8f5cc02e042 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.872669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3552c-51ed-4330-b78b-4ac93d128571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.873334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3552c-51ed-4330-b78b-4ac93d128571 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.873714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3552c-51ed-4330-b78b-4ac93d128571 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.873996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3552c-51ed-4330-b78b-4ac93d128571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.877526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.883450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffc9a6-b636-49fc-86a2-cd41c201a747 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.887276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3552c-51ed-4330-b78b-4ac93d128571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.888731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3552c-51ed-4330-b78b-4ac93d128571 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-66f3552c-51ed-4330-b78b-4ac93d128571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:52.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:52.905187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:52.906151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:52.906151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:52.906151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:52.906151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 357/711] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:52.907863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd1ac7-e752-443e-935f-ea9cc4ad2d0f None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.908507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd1ac7-e752-443e-935f-ea9cc4ad2d0f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.908507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd1ac7-e752-443e-935f-ea9cc4ad2d0f None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.913701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 358/712] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:52.915003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525baf3-8cce-439b-be43-c3933fb5c6b9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.915282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525baf3-8cce-439b-be43-c3933fb5c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.916155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525baf3-8cce-439b-be43-c3933fb5c6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.931193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3ef63-2b1b-4d6d-89c2-eac947b46a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.931802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3ef63-2b1b-4d6d-89c2-eac947b46a73 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.932084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3ef63-2b1b-4d6d-89c2-eac947b46a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.932354 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3ef63-2b1b-4d6d-89c2-eac947b46a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.933114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3ef63-2b1b-4d6d-89c2-eac947b46a73 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=c53b9bcd675940c19febbc8fefb1f28b) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:52.933417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3ef63-2b1b-4d6d-89c2-eac947b46a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.934832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3ef63-2b1b-4d6d-89c2-eac947b46a73 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.936063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3525baf3-8cce-439b-be43-c3933fb5c6b9 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:52.938362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3525baf3-8cce-439b-be43-c3933fb5c6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:52.949264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 355/713] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/role_assignments?user.id=c53b9bcd675940c19febbc8fefb1f28b&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:52.950380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3ef63-2b1b-4d6d-89c2-eac947b46a73 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.974917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.033649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 359/714] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:53.130153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3ef63-2b1b-4d6d-89c2-eac947b46a73 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.174873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de43668-d515-4e0c-9858-2ef13e13a3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.175490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43668-d515-4e0c-9858-2ef13e13a3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.175770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43668-d515-4e0c-9858-2ef13e13a3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.176037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43668-d515-4e0c-9858-2ef13e13a3e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.176692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de43668-d515-4e0c-9858-2ef13e13a3e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:53.176994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de43668-d515-4e0c-9858-2ef13e13a3e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.178424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de43668-d515-4e0c-9858-2ef13e13a3e4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.197177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 356/715] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:53.199439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525baf3-8cce-439b-be43-c3933fb5c6b9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.315978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525baf3-8cce-439b-be43-c3933fb5c6b9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.331485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.346295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de43668-d515-4e0c-9858-2ef13e13a3e4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.353146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd36997-5faa-4aa8-ac5b-0942f1b50aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.353762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd36997-5faa-4aa8-ac5b-0942f1b50aea None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.356348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd36997-5faa-4aa8-ac5b-0942f1b50aea None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.356348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd36997-5faa-4aa8-ac5b-0942f1b50aea None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.356348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd36997-5faa-4aa8-ac5b-0942f1b50aea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.356602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd36997-5faa-4aa8-ac5b-0942f1b50aea None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.388310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 360/716] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:53 2022] POST /identity/v3/users => generated 254 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:53.452794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.483412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de43668-d515-4e0c-9858-2ef13e13a3e4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.523302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e300b-f51f-4b28-94ed-1c760e84736b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.523963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e300b-f51f-4b28-94ed-1c760e84736b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.524203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e300b-f51f-4b28-94ed-1c760e84736b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.524476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e300b-f51f-4b28-94ed-1c760e84736b None admin] PATH_INFO: `/v3/services/network` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.525061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e300b-f51f-4b28-94ed-1c760e84736b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.526515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e300b-f51f-4b28-94ed-1c760e84736b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-f52e300b-f51f-4b28-94ed-1c760e84736b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:53.540288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:53.541375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:53.542795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 357/717] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:53.547919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd36997-5faa-4aa8-ac5b-0942f1b50aea None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.679350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd36997-5faa-4aa8-ac5b-0942f1b50aea None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.716478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f6177-990d-4a5b-8fd8-78139cb1b4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.717469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f6177-990d-4a5b-8fd8-78139cb1b4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.717976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f6177-990d-4a5b-8fd8-78139cb1b4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.718476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f6177-990d-4a5b-8fd8-78139cb1b4e1 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.719491 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f6177-990d-4a5b-8fd8-78139cb1b4e1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:53.719993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f6177-990d-4a5b-8fd8-78139cb1b4e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.721665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f6177-990d-4a5b-8fd8-78139cb1b4e1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.726968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 361/718] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:53.732351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e300b-f51f-4b28-94ed-1c760e84736b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.845986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e300b-f51f-4b28-94ed-1c760e84736b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.884202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7e1ac-f222-438e-91f9-b8af086268aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.885206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e1ac-f222-438e-91f9-b8af086268aa None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.885734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e1ac-f222-438e-91f9-b8af086268aa None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.886216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e1ac-f222-438e-91f9-b8af086268aa None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.887053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7e1ac-f222-438e-91f9-b8af086268aa None admin] RBAC: Adding query filter params (type=network) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:53.887555 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7e1ac-f222-438e-91f9-b8af086268aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.889151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7e1ac-f222-438e-91f9-b8af086268aa None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.896583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 358/719] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:54.826282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925639b-4e85-4679-ab12-e795b9f6b664 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:54.826807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925639b-4e85-4679-ab12-e795b9f6b664 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:54.826807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925639b-4e85-4679-ab12-e795b9f6b664 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:54.829067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 362/720] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:54.834338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177be719-fccc-44aa-b72a-c15ee0527d9d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:54.834752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177be719-fccc-44aa-b72a-c15ee0527d9d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:54.835010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177be719-fccc-44aa-b72a-c15ee0527d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:54.841381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:54.855270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-177be719-fccc-44aa-b72a-c15ee0527d9d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:54.856476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-177be719-fccc-44aa-b72a-c15ee0527d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:54.953992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 359/721] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:55.164552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc423b4d-6611-4aba-91a9-2616c1cbb816 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:55.165258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc423b4d-6611-4aba-91a9-2616c1cbb816 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:55.165735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc423b4d-6611-4aba-91a9-2616c1cbb816 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:55.167512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 363/722] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:55.173174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3013865-0003-486e-b6e5-c34c31127797 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:55.173681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3013865-0003-486e-b6e5-c34c31127797 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:55.174148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3013865-0003-486e-b6e5-c34c31127797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:55.202026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d3013865-0003-486e-b6e5-c34c31127797 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:55.203695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d3013865-0003-486e-b6e5-c34c31127797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:55.301824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 360/723] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:55.358359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc423b4d-6611-4aba-91a9-2616c1cbb816 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:55.402336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:55.493614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc423b4d-6611-4aba-91a9-2616c1cbb816 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:55.543248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae648909-dded-468f-a8dd-5ee91aa762a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:55.543887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae648909-dded-468f-a8dd-5ee91aa762a5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:55.544164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae648909-dded-468f-a8dd-5ee91aa762a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:55.544484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae648909-dded-468f-a8dd-5ee91aa762a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:55.559710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae648909-dded-468f-a8dd-5ee91aa762a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ae648909-dded-468f-a8dd-5ee91aa762a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:55.565483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:55.566394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:55.567214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:55.569438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 364/724] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:55.578292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3013865-0003-486e-b6e5-c34c31127797 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:55.609614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd41e50-8e57-4f9b-88de-3b31b085986c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:55.610067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd41e50-8e57-4f9b-88de-3b31b085986c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:55.610348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd41e50-8e57-4f9b-88de-3b31b085986c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:55.612375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 365/725] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:55.628142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b258e-2210-450f-85ab-199e43d0bbc2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:55.628425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b258e-2210-450f-85ab-199e43d0bbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:55.628698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b258e-2210-450f-85ab-199e43d0bbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:55.707245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:55.774405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3013865-0003-486e-b6e5-c34c31127797 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:55.811525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7402b-2045-4354-9518-54517c8eb721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:55.813184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7402b-2045-4354-9518-54517c8eb721 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:55.813747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7402b-2045-4354-9518-54517c8eb721 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:55.814348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7402b-2045-4354-9518-54517c8eb721 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:55.815219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7402b-2045-4354-9518-54517c8eb721 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:55.815725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7402b-2045-4354-9518-54517c8eb721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:55.817320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7402b-2045-4354-9518-54517c8eb721 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:55.826933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 361/726] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:55.832449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7402b-2045-4354-9518-54517c8eb721 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:55.838839 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f46b258e-2210-450f-85ab-199e43d0bbc2 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:55.840589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f46b258e-2210-450f-85ab-199e43d0bbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:55.921211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 366/727] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:55.922746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b258e-2210-450f-85ab-199e43d0bbc2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:55.957439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:55.961759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7402b-2045-4354-9518-54517c8eb721 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.003132 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.024576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e71ca7-b01c-40ca-ad1d-32fda5c71958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.025226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e71ca7-b01c-40ca-ad1d-32fda5c71958 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.025483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e71ca7-b01c-40ca-ad1d-32fda5c71958 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.025757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e71ca7-b01c-40ca-ad1d-32fda5c71958 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.026423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e71ca7-b01c-40ca-ad1d-32fda5c71958 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:56.027773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e71ca7-b01c-40ca-ad1d-32fda5c71958 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:56.034688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b258e-2210-450f-85ab-199e43d0bbc2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.079577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 362/728] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:48:55 2022] POST /identity/v3/services => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:56.090094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528baa5-19ca-4a6f-a9dc-7ef72e071d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.091130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528baa5-19ca-4a6f-a9dc-7ef72e071d2f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.091599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528baa5-19ca-4a6f-a9dc-7ef72e071d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.092074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528baa5-19ca-4a6f-a9dc-7ef72e071d2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.104206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528baa5-19ca-4a6f-a9dc-7ef72e071d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:56.106202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528baa5-19ca-4a6f-a9dc-7ef72e071d2f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-0528baa5-19ca-4a6f-a9dc-7ef72e071d2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:56.124007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:56.125927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:56.126830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:56.126830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:56.126830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:56.129668 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fe0e5-707e-488d-9481-603a9c1f6168 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.130305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fe0e5-707e-488d-9481-603a9c1f6168 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.130771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fe0e5-707e-488d-9481-603a9c1f6168 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.132541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 363/729] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:56.133855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 367/730] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:56.139888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fe0e5-707e-488d-9481-603a9c1f6168 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.144624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7449f66-ce16-4221-864d-6007f09a57fc None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.144904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7449f66-ce16-4221-864d-6007f09a57fc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.145211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7449f66-ce16-4221-864d-6007f09a57fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.164228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f7449f66-ce16-4221-864d-6007f09a57fc None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:56.165438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f7449f66-ce16-4221-864d-6007f09a57fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:56.241977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 368/731] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:56.286561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7449f66-ce16-4221-864d-6007f09a57fc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.289509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.338408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fe0e5-707e-488d-9481-603a9c1f6168 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.378587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055001f-7b6a-4fa0-af4b-d466beff7686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.379370 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055001f-7b6a-4fa0-af4b-d466beff7686 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.380334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055001f-7b6a-4fa0-af4b-d466beff7686 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.380334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055001f-7b6a-4fa0-af4b-d466beff7686 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.380607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055001f-7b6a-4fa0-af4b-d466beff7686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:56.380912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055001f-7b6a-4fa0-af4b-d466beff7686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:56.382375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055001f-7b6a-4fa0-af4b-d466beff7686 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:56.398856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 364/732] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:56.399549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c9cd6-3bd1-46ee-9b75-84185364a836 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.401195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c9cd6-3bd1-46ee-9b75-84185364a836 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.401195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c9cd6-3bd1-46ee-9b75-84185364a836 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.401596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 365/733] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:56.407005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dfefd-7edd-4f81-b44d-15af87032b1e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.407309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dfefd-7edd-4f81-b44d-15af87032b1e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.407614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dfefd-7edd-4f81-b44d-15af87032b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.420089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7449f66-ce16-4221-864d-6007f09a57fc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.428424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-102dfefd-7edd-4f81-b44d-15af87032b1e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:56.429733 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-102dfefd-7edd-4f81-b44d-15af87032b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:56.460874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83767c-f21e-47b5-9411-479052bcf6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.461867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83767c-f21e-47b5-9411-479052bcf6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.462313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83767c-f21e-47b5-9411-479052bcf6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.462860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83767c-f21e-47b5-9411-479052bcf6e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.475166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83767c-f21e-47b5-9411-479052bcf6e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:56.478508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-eb83767c-f21e-47b5-9411-479052bcf6e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:56.482830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:56.484719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:56.485793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:56.489032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 369/734] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:56.490123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83767c-f21e-47b5-9411-479052bcf6e2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.520712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 366/735] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:56.529963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102dfefd-7edd-4f81-b44d-15af87032b1e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.578462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.621500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83767c-f21e-47b5-9411-479052bcf6e2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.635724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102dfefd-7edd-4f81-b44d-15af87032b1e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.658145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c388612-01d7-423b-807d-29f692c97eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.659120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c388612-01d7-423b-807d-29f692c97eec None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.659620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c388612-01d7-423b-807d-29f692c97eec None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.660095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c388612-01d7-423b-807d-29f692c97eec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.667616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c388612-01d7-423b-807d-29f692c97eec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:56.669378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c388612-01d7-423b-807d-29f692c97eec None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:56.674244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218bc79-aa9f-48f3-9b83-6f34d9e6271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.674854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218bc79-aa9f-48f3-9b83-6f34d9e6271a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.675131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218bc79-aa9f-48f3-9b83-6f34d9e6271a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.675396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218bc79-aa9f-48f3-9b83-6f34d9e6271a None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.676062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218bc79-aa9f-48f3-9b83-6f34d9e6271a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:56.676359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218bc79-aa9f-48f3-9b83-6f34d9e6271a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:56.678085 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218bc79-aa9f-48f3-9b83-6f34d9e6271a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-0c388612-01d7-423b-807d-29f692c97eec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:56.684788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:56.692722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:48:56.693727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:56.694526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 367/737] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:56.694526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218bc79-aa9f-48f3-9b83-6f34d9e6271a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.694666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 370/737] 10.222.0.78 () {56 vars in 1093 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:56.694666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c388612-01d7-423b-807d-29f692c97eec None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.733310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.801683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218bc79-aa9f-48f3-9b83-6f34d9e6271a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.808403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c388612-01d7-423b-807d-29f692c97eec None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.834108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.839023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dc0e3-0a0d-48f5-8cf9-29de9a34ab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.839893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dc0e3-0a0d-48f5-8cf9-29de9a34ab6b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.840363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dc0e3-0a0d-48f5-8cf9-29de9a34ab6b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.840923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dc0e3-0a0d-48f5-8cf9-29de9a34ab6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.846297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e992-be9a-49e6-83f7-e4280ba9f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.846900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e992-be9a-49e6-83f7-e4280ba9f6af None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.847253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e992-be9a-49e6-83f7-e4280ba9f6af None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.847619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e992-be9a-49e6-83f7-e4280ba9f6af None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.848150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e992-be9a-49e6-83f7-e4280ba9f6af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:56.848448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e992-be9a-49e6-83f7-e4280ba9f6af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:56.849904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e992-be9a-49e6-83f7-e4280ba9f6af None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:56.854304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36dc0e3-0a0d-48f5-8cf9-29de9a34ab6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-e36dc0e3-0a0d-48f5-8cf9-29de9a34ab6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:56.860650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:56.861603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:48:56.862435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:56.868151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 368/738] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:56.868151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dc0e3-0a0d-48f5-8cf9-29de9a34ab6b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.872163 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 371/739] 10.222.0.78 () {56 vars in 1133 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:56.873023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e992-be9a-49e6-83f7-e4280ba9f6af None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.970693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.977094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e992-be9a-49e6-83f7-e4280ba9f6af None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.977239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dc0e3-0a0d-48f5-8cf9-29de9a34ab6b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.012689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88b5f6-84e9-476a-b594-aa69a5e48a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.013358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88b5f6-84e9-476a-b594-aa69a5e48a74 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.013641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88b5f6-84e9-476a-b594-aa69a5e48a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.013911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88b5f6-84e9-476a-b594-aa69a5e48a74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.014732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73524d5c-afdf-4f7d-9ebf-6bad20d56d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.015512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73524d5c-afdf-4f7d-9ebf-6bad20d56d33 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.015845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73524d5c-afdf-4f7d-9ebf-6bad20d56d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.016148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73524d5c-afdf-4f7d-9ebf-6bad20d56d33 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.017179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73524d5c-afdf-4f7d-9ebf-6bad20d56d33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.018027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73524d5c-afdf-4f7d-9ebf-6bad20d56d33 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.026563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88b5f6-84e9-476a-b594-aa69a5e48a74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.027990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88b5f6-84e9-476a-b594-aa69a5e48a74 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.037133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 372/740] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:48:56 2022] POST /identity/v3/services => generated 240 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:57.037133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73524d5c-afdf-4f7d-9ebf-6bad20d56d33 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-dd88b5f6-84e9-476a-b594-aa69a5e48a74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:57.043020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:57.044074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:57.045036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:57.045036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:57.045036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:57.045036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 369/741] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:57.045346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88b5f6-84e9-476a-b594-aa69a5e48a74 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.094901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:57.177517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88b5f6-84e9-476a-b594-aa69a5e48a74 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.238633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73524d5c-afdf-4f7d-9ebf-6bad20d56d33 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.250506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31cd82-4539-43e6-ac37-d75b444aedcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.251370 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31cd82-4539-43e6-ac37-d75b444aedcb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.252909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31cd82-4539-43e6-ac37-d75b444aedcb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.252909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31cd82-4539-43e6-ac37-d75b444aedcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.268789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31cd82-4539-43e6-ac37-d75b444aedcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.270197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31cd82-4539-43e6-ac37-d75b444aedcb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.286441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-bb31cd82-4539-43e6-ac37-d75b444aedcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:57.298437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:57.299444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:57.300311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:57.300311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:57.300311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:57.302988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 370/742] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:57.304591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31cd82-4539-43e6-ac37-d75b444aedcb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.314339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0f361-fba5-46ca-806d-48fa2d4251a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.315034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0f361-fba5-46ca-806d-48fa2d4251a6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.315411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0f361-fba5-46ca-806d-48fa2d4251a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.315774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0f361-fba5-46ca-806d-48fa2d4251a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.316562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0f361-fba5-46ca-806d-48fa2d4251a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:57.316955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0f361-fba5-46ca-806d-48fa2d4251a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.320920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0f361-fba5-46ca-806d-48fa2d4251a6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.334206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 373/743] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:57.335247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0f361-fba5-46ca-806d-48fa2d4251a6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.455158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31cd82-4539-43e6-ac37-d75b444aedcb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.502874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:57.505906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3943-b121-4ec0-a9bf-847183f9e48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.506835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3943-b121-4ec0-a9bf-847183f9e48f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.509262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0f361-fba5-46ca-806d-48fa2d4251a6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.509377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3943-b121-4ec0-a9bf-847183f9e48f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.509377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3943-b121-4ec0-a9bf-847183f9e48f None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.509377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3943-b121-4ec0-a9bf-847183f9e48f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:57.509377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3943-b121-4ec0-a9bf-847183f9e48f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.511374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3943-b121-4ec0-a9bf-847183f9e48f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.534491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 371/744] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:57.535604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3943-b121-4ec0-a9bf-847183f9e48f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.575766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c93874-ec6b-43e8-b41f-4109da600089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.579103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c93874-ec6b-43e8-b41f-4109da600089 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.579262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c93874-ec6b-43e8-b41f-4109da600089 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.579531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c93874-ec6b-43e8-b41f-4109da600089 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.582847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c93874-ec6b-43e8-b41f-4109da600089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:57.583167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c93874-ec6b-43e8-b41f-4109da600089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.584549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c93874-ec6b-43e8-b41f-4109da600089 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.607319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:57.621574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 374/745] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:57.623222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c93874-ec6b-43e8-b41f-4109da600089 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.658778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3943-b121-4ec0-a9bf-847183f9e48f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.725920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20210101-6c50-430d-8ef7-63f570ab8c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.726646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20210101-6c50-430d-8ef7-63f570ab8c68 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.726839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20210101-6c50-430d-8ef7-63f570ab8c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.727076 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20210101-6c50-430d-8ef7-63f570ab8c68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.753160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20210101-6c50-430d-8ef7-63f570ab8c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.754625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20210101-6c50-430d-8ef7-63f570ab8c68 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.775498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c93874-ec6b-43e8-b41f-4109da600089 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-20210101-6c50-430d-8ef7-63f570ab8c68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:57.789984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:57.790975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:57.791859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:57.791859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:57.791859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:57.795834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 372/746] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:57.797426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20210101-6c50-430d-8ef7-63f570ab8c68 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.823885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56988afe-57c5-4582-9baa-d23771487080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.825201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56988afe-57c5-4582-9baa-d23771487080 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.825754 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56988afe-57c5-4582-9baa-d23771487080 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.826404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56988afe-57c5-4582-9baa-d23771487080 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.838996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56988afe-57c5-4582-9baa-d23771487080 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.840709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56988afe-57c5-4582-9baa-d23771487080 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-56988afe-57c5-4582-9baa-d23771487080 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:57.856436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:57.857850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:48:57.858770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:57.867182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 375/747] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:57.867182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56988afe-57c5-4582-9baa-d23771487080 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.899301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:57.935659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20210101-6c50-430d-8ef7-63f570ab8c68 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.959375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.000621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb13c68-c7af-4730-8020-d3eb314847f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.002064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13c68-c7af-4730-8020-d3eb314847f3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.002574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13c68-c7af-4730-8020-d3eb314847f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.003059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13c68-c7af-4730-8020-d3eb314847f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.012568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb13c68-c7af-4730-8020-d3eb314847f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.014594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb13c68-c7af-4730-8020-d3eb314847f3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.026448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56988afe-57c5-4582-9baa-d23771487080 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ceb13c68-c7af-4730-8020-d3eb314847f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:58.031961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:58.032814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:58.033737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:58.033737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:58.033737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:58.033737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:58.033737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:58.033737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:58.033737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:58.033737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:58.036104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 373/748] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:58.037196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb13c68-c7af-4730-8020-d3eb314847f3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.085551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed27f41-ff68-4aab-8bec-056f3fe7b013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.086770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed27f41-ff68-4aab-8bec-056f3fe7b013 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.087132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed27f41-ff68-4aab-8bec-056f3fe7b013 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.087664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed27f41-ff68-4aab-8bec-056f3fe7b013 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.088803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed27f41-ff68-4aab-8bec-056f3fe7b013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:58.089389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed27f41-ff68-4aab-8bec-056f3fe7b013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.092165 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed27f41-ff68-4aab-8bec-056f3fe7b013 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.121226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 376/749] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:58.122765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed27f41-ff68-4aab-8bec-056f3fe7b013 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.157977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb13c68-c7af-4730-8020-d3eb314847f3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.196312 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45aab7-99f1-4b47-8f75-b15d324fee12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.197348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45aab7-99f1-4b47-8f75-b15d324fee12 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.197882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45aab7-99f1-4b47-8f75-b15d324fee12 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.198380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45aab7-99f1-4b47-8f75-b15d324fee12 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.199305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45aab7-99f1-4b47-8f75-b15d324fee12 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:58.199813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45aab7-99f1-4b47-8f75-b15d324fee12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.201622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45aab7-99f1-4b47-8f75-b15d324fee12 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.221631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 374/750] 10.222.0.78 () {56 vars in 1135 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:58.227156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45aab7-99f1-4b47-8f75-b15d324fee12 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.260767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.261982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.300512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed27f41-ff68-4aab-8bec-056f3fe7b013 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.337086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf70ab-5ff3-4b61-85c3-eced9072748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.338094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf70ab-5ff3-4b61-85c3-eced9072748b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.338388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf70ab-5ff3-4b61-85c3-eced9072748b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.338656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf70ab-5ff3-4b61-85c3-eced9072748b None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.339334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf70ab-5ff3-4b61-85c3-eced9072748b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:58.339640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf70ab-5ff3-4b61-85c3-eced9072748b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.341108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf70ab-5ff3-4b61-85c3-eced9072748b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.351870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45aab7-99f1-4b47-8f75-b15d324fee12 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.354043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 377/751] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:58.355013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf70ab-5ff3-4b61-85c3-eced9072748b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.386987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f717eb8-79e9-4b8a-93f9-f6a27a63d799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.387619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f717eb8-79e9-4b8a-93f9-f6a27a63d799 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.387900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f717eb8-79e9-4b8a-93f9-f6a27a63d799 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.388177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f717eb8-79e9-4b8a-93f9-f6a27a63d799 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.396104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f717eb8-79e9-4b8a-93f9-f6a27a63d799 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.397571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f717eb8-79e9-4b8a-93f9-f6a27a63d799 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9f717eb8-79e9-4b8a-93f9-f6a27a63d799 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:48:58.412126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:58.413127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:58.414099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 375/752] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:58.415192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfec0be-7188-4bc4-a103-7ab2e53f1105 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.415192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfec0be-7188-4bc4-a103-7ab2e53f1105 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.415192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfec0be-7188-4bc4-a103-7ab2e53f1105 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.416471 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 376/753] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:58.420895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfec0be-7188-4bc4-a103-7ab2e53f1105 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.473189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf70ab-5ff3-4b61-85c3-eced9072748b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.510264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba8b2d-353b-40a2-886b-83322c838547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.510786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba8b2d-353b-40a2-886b-83322c838547 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.511066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba8b2d-353b-40a2-886b-83322c838547 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.511336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba8b2d-353b-40a2-886b-83322c838547 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.522045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.525031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.539170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfec0be-7188-4bc4-a103-7ab2e53f1105 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.578008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc1d23-ce99-4e0a-8cfb-a4ec1170ca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.578963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc1d23-ce99-4e0a-8cfb-a4ec1170ca4a None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.579413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc1d23-ce99-4e0a-8cfb-a4ec1170ca4a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.580139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc1d23-ce99-4e0a-8cfb-a4ec1170ca4a None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/f9b88c495fcb47a289a6a45276c891e0/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.582239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc1d23-ce99-4e0a-8cfb-a4ec1170ca4a None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=f9b88c495fcb47a289a6a45276c891e0, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.583824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc1d23-ce99-4e0a-8cfb-a4ec1170ca4a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.583932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba8b2d-353b-40a2-886b-83322c838547 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.585337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba8b2d-353b-40a2-886b-83322c838547 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.599406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 377/754] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:58 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/f9b88c495fcb47a289a6a45276c891e0/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:58.600212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc1d23-ce99-4e0a-8cfb-a4ec1170ca4a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.727064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc1d23-ce99-4e0a-8cfb-a4ec1170ca4a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.733002 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 378/755] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:48:58.734730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b3e02-f664-491b-bd32-ec1e8502e5e1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.735231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b3e02-f664-491b-bd32-ec1e8502e5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.735679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b3e02-f664-491b-bd32-ec1e8502e5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.758405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b92b3e02-f664-491b-bd32-ec1e8502e5e1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:58.760072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b92b3e02-f664-491b-bd32-ec1e8502e5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:58.772656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169406e-555f-48f2-90ca-232769d2169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.773234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169406e-555f-48f2-90ca-232769d2169a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.773515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169406e-555f-48f2-90ca-232769d2169a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.773785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169406e-555f-48f2-90ca-232769d2169a None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.774521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169406e-555f-48f2-90ca-232769d2169a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:58.774826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169406e-555f-48f2-90ca-232769d2169a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.776259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169406e-555f-48f2-90ca-232769d2169a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.797499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.797934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 378/756] 10.222.0.78 () {56 vars in 1149 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:58.799785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169406e-555f-48f2-90ca-232769d2169a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.812848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.857814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 379/757] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:48:58.858735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b3e02-f664-491b-bd32-ec1e8502e5e1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.974241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b3e02-f664-491b-bd32-ec1e8502e5e1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.003935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169406e-555f-48f2-90ca-232769d2169a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.013401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0f5b5-a6ed-4257-a76f-88ff34ec8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.014012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0f5b5-a6ed-4257-a76f-88ff34ec8684 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.014286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0f5b5-a6ed-4257-a76f-88ff34ec8684 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.014560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0f5b5-a6ed-4257-a76f-88ff34ec8684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.052857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.091249 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0f5b5-a6ed-4257-a76f-88ff34ec8684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.093065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0f5b5-a6ed-4257-a76f-88ff34ec8684 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.101778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404039d-feaa-4abc-8d2d-fd9ef9f6851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.102521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404039d-feaa-4abc-8d2d-fd9ef9f6851f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.102881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404039d-feaa-4abc-8d2d-fd9ef9f6851f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.103223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404039d-feaa-4abc-8d2d-fd9ef9f6851f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.138143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404039d-feaa-4abc-8d2d-fd9ef9f6851f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.139705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404039d-feaa-4abc-8d2d-fd9ef9f6851f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a404039d-feaa-4abc-8d2d-fd9ef9f6851f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:59.171904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:59.173055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:59.173974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:59.173974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:59.173974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:59.181446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 379/758] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:59.182375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404039d-feaa-4abc-8d2d-fd9ef9f6851f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.207493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 380/759] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:48:59.208688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0f5b5-a6ed-4257-a76f-88ff34ec8684 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.259525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.301678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.315139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404039d-feaa-4abc-8d2d-fd9ef9f6851f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.346585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0f5b5-a6ed-4257-a76f-88ff34ec8684 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.356543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464cc3d-a7be-471b-90fd-fddb699ba58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.357178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464cc3d-a7be-471b-90fd-fddb699ba58b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.357453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464cc3d-a7be-471b-90fd-fddb699ba58b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.357716 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464cc3d-a7be-471b-90fd-fddb699ba58b None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.358345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464cc3d-a7be-471b-90fd-fddb699ba58b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:59.358642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464cc3d-a7be-471b-90fd-fddb699ba58b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.360685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464cc3d-a7be-471b-90fd-fddb699ba58b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.372573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 380/760] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.373441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464cc3d-a7be-471b-90fd-fddb699ba58b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.386409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c85a0-b5b9-4cb6-8f5b-58c66084ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.387036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c85a0-b5b9-4cb6-8f5b-58c66084ebec None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.387315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c85a0-b5b9-4cb6-8f5b-58c66084ebec None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.387586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c85a0-b5b9-4cb6-8f5b-58c66084ebec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.388162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c85a0-b5b9-4cb6-8f5b-58c66084ebec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.389567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c85a0-b5b9-4cb6-8f5b-58c66084ebec None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-237c85a0-b5b9-4cb6-8f5b-58c66084ebec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:59.398462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:59.399447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:59.400299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 381/761] 10.222.0.78 () {56 vars in 1129 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:59.401114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17a970-c51b-4d3e-97a8-65877d07a675 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.401114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17a970-c51b-4d3e-97a8-65877d07a675 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.401790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17a970-c51b-4d3e-97a8-65877d07a675 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.405130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 382/762] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:48:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:59.405130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17a970-c51b-4d3e-97a8-65877d07a675 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.487128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464cc3d-a7be-471b-90fd-fddb699ba58b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.508896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.519382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17a970-c51b-4d3e-97a8-65877d07a675 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.526127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588c4a6-f3f8-4f0e-93e5-32a1448c1187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.526783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588c4a6-f3f8-4f0e-93e5-32a1448c1187 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.527070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588c4a6-f3f8-4f0e-93e5-32a1448c1187 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.527345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588c4a6-f3f8-4f0e-93e5-32a1448c1187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.538734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588c4a6-f3f8-4f0e-93e5-32a1448c1187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.540190 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588c4a6-f3f8-4f0e-93e5-32a1448c1187 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.550911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.556085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326c607-4702-471e-944a-ccf66e6b70ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7588c4a6-f3f8-4f0e-93e5-32a1448c1187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:59.556940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:59.557888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:59.558780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:59.558780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:59.558780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:48:59.558959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326c607-4702-471e-944a-ccf66e6b70ce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.558959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326c607-4702-471e-944a-ccf66e6b70ce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.558959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326c607-4702-471e-944a-ccf66e6b70ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.561219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 381/763] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:59.562153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588c4a6-f3f8-4f0e-93e5-32a1448c1187 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.567818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326c607-4702-471e-944a-ccf66e6b70ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.569481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326c607-4702-471e-944a-ccf66e6b70ce None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b326c607-4702-471e-944a-ccf66e6b70ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:48:59.580620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:59.581530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:59.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:48:59.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:48:59.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:48:59.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:48:59.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:48:59.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:48:59.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:48:59.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:48:59.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 383/764] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:59.583463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a577596-9dae-40b2-aab4-d93837c57488 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.583463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a577596-9dae-40b2-aab4-d93837c57488 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.583621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a577596-9dae-40b2-aab4-d93837c57488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.602137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3a577596-9dae-40b2-aab4-d93837c57488 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:59.603421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3a577596-9dae-40b2-aab4-d93837c57488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:59.681118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 384/765] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:48:59.681330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588c4a6-f3f8-4f0e-93e5-32a1448c1187 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.681416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a577596-9dae-40b2-aab4-d93837c57488 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.721280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b750d46-ae54-4e90-bab7-d6d89a8a3320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.721999 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b750d46-ae54-4e90-bab7-d6d89a8a3320 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.722275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b750d46-ae54-4e90-bab7-d6d89a8a3320 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.722541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b750d46-ae54-4e90-bab7-d6d89a8a3320 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.723275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b750d46-ae54-4e90-bab7-d6d89a8a3320 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:59.723576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b750d46-ae54-4e90-bab7-d6d89a8a3320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.724934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b750d46-ae54-4e90-bab7-d6d89a8a3320 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.735056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 382/766] 10.222.0.78 () {56 vars in 1124 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.736065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b750d46-ae54-4e90-bab7-d6d89a8a3320 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.775782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.802999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a577596-9dae-40b2-aab4-d93837c57488 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.834231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.850536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6ccb1-efd5-4303-86b2-c5993b6ad3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.851556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6ccb1-efd5-4303-86b2-c5993b6ad3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.851908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6ccb1-efd5-4303-86b2-c5993b6ad3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.852251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6ccb1-efd5-4303-86b2-c5993b6ad3b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.853099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6ccb1-efd5-4303-86b2-c5993b6ad3b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:59.853483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6ccb1-efd5-4303-86b2-c5993b6ad3b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.853803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b750d46-ae54-4e90-bab7-d6d89a8a3320 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.855065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6ccb1-efd5-4303-86b2-c5993b6ad3b1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.877237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 385/767] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.878766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6ccb1-efd5-4303-86b2-c5993b6ad3b1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.897237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf73b93-1378-4c5f-abb7-d059db427ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.898056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf73b93-1378-4c5f-abb7-d059db427ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.898504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf73b93-1378-4c5f-abb7-d059db427ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.898915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf73b93-1378-4c5f-abb7-d059db427ab4 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.899793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf73b93-1378-4c5f-abb7-d059db427ab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:59.900257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf73b93-1378-4c5f-abb7-d059db427ab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.901880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf73b93-1378-4c5f-abb7-d059db427ab4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.916736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 383/768] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.917820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf73b93-1378-4c5f-abb7-d059db427ab4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.003431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6ccb1-efd5-4303-86b2-c5993b6ad3b1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.029336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.036392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf73b93-1378-4c5f-abb7-d059db427ab4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.039538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ee2b5-d3b5-4306-b548-27e80a09d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.040133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ee2b5-d3b5-4306-b548-27e80a09d0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.040400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ee2b5-d3b5-4306-b548-27e80a09d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.040667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ee2b5-d3b5-4306-b548-27e80a09d0f9 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.041323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ee2b5-d3b5-4306-b548-27e80a09d0f9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:00.041620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ee2b5-d3b5-4306-b548-27e80a09d0f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.042975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ee2b5-d3b5-4306-b548-27e80a09d0f9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.052443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 386/769] 10.222.0.78 () {56 vars in 1124 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.052443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ee2b5-d3b5-4306-b548-27e80a09d0f9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.078328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb872f-4b52-4255-b7bf-d474b197274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.079163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb872f-4b52-4255-b7bf-d474b197274a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.079445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb872f-4b52-4255-b7bf-d474b197274a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.079717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb872f-4b52-4255-b7bf-d474b197274a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.086947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb872f-4b52-4255-b7bf-d474b197274a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.088537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb872f-4b52-4255-b7bf-d474b197274a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7bbb872f-4b52-4255-b7bf-d474b197274a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:00.100070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:00.100943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:00.101890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:00.101890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:00.101890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:00.101890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:00.101890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:00.101890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:00.101890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:00.101890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:00.101890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 384/770] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:00.102377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb872f-4b52-4255-b7bf-d474b197274a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.103883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.167689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ee2b5-d3b5-4306-b548-27e80a09d0f9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.203205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69e6fe-a7b4-42c9-94b9-af79e6b25873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.203809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69e6fe-a7b4-42c9-94b9-af79e6b25873 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.204098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69e6fe-a7b4-42c9-94b9-af79e6b25873 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.204353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69e6fe-a7b4-42c9-94b9-af79e6b25873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.205094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69e6fe-a7b4-42c9-94b9-af79e6b25873 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=c53b9bcd675940c19febbc8fefb1f28b) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:00.205408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69e6fe-a7b4-42c9-94b9-af79e6b25873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.206829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69e6fe-a7b4-42c9-94b9-af79e6b25873 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.212748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 387/771] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/role_assignments?user.id=c53b9bcd675940c19febbc8fefb1f28b&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.213470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69e6fe-a7b4-42c9-94b9-af79e6b25873 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.227151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb872f-4b52-4255-b7bf-d474b197274a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.275896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd2dee-6875-483f-b344-f8576e29a320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.276547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd2dee-6875-483f-b344-f8576e29a320 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.276823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd2dee-6875-483f-b344-f8576e29a320 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.277134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd2dee-6875-483f-b344-f8576e29a320 None admin] PATH_INFO: `/v3/services/network` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.277734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd2dee-6875-483f-b344-f8576e29a320 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.279464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd2dee-6875-483f-b344-f8576e29a320 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-8cbd2dee-6875-483f-b344-f8576e29a320 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:00.289695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:00.290754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:00.291596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:00.295915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 385/772] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:00.297428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd2dee-6875-483f-b344-f8576e29a320 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.329482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.362145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69e6fe-a7b4-42c9-94b9-af79e6b25873 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.378360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.413485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787dd80-4cda-4488-8d57-4ab38271af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.414586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787dd80-4cda-4488-8d57-4ab38271af22 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.415142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787dd80-4cda-4488-8d57-4ab38271af22 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.415422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787dd80-4cda-4488-8d57-4ab38271af22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.416379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787dd80-4cda-4488-8d57-4ab38271af22 None admin] RBAC: Adding query filter params (interface=public, service_id=71882d685e2f4133bc8ca5bd4c98d0e1, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:00.417066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787dd80-4cda-4488-8d57-4ab38271af22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.418744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787dd80-4cda-4488-8d57-4ab38271af22 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.419665 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd2dee-6875-483f-b344-f8576e29a320 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.429603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 388/773] 10.222.0.78 () {56 vars in 1251 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/endpoints?service_id=71882d685e2f4133bc8ca5bd4c98d0e1&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.430458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787dd80-4cda-4488-8d57-4ab38271af22 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.463001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3c50a-d57a-4e5f-8f52-1fe630f04b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.463641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3c50a-d57a-4e5f-8f52-1fe630f04b0d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.463923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3c50a-d57a-4e5f-8f52-1fe630f04b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.464189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3c50a-d57a-4e5f-8f52-1fe630f04b0d None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.464934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3c50a-d57a-4e5f-8f52-1fe630f04b0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:00.465266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3c50a-d57a-4e5f-8f52-1fe630f04b0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.466681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3c50a-d57a-4e5f-8f52-1fe630f04b0d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.480261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 386/774] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.481316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3c50a-d57a-4e5f-8f52-1fe630f04b0d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.565643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787dd80-4cda-4488-8d57-4ab38271af22 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.575456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.602045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06efe8c-3ad6-4633-928d-e2281456923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.602696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06efe8c-3ad6-4633-928d-e2281456923e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.602975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06efe8c-3ad6-4633-928d-e2281456923e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.603240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06efe8c-3ad6-4633-928d-e2281456923e None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.603860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06efe8c-3ad6-4633-928d-e2281456923e None admin] RBAC: Adding query filter params (name=network) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:00.604154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06efe8c-3ad6-4633-928d-e2281456923e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.604904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3c50a-d57a-4e5f-8f52-1fe630f04b0d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.605558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06efe8c-3ad6-4633-928d-e2281456923e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.615157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 389/775] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.615951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06efe8c-3ad6-4633-928d-e2281456923e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.650205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b88fe-4081-4b7e-86ce-41d792dd8a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.650816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b88fe-4081-4b7e-86ce-41d792dd8a08 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.651100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b88fe-4081-4b7e-86ce-41d792dd8a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.651376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b88fe-4081-4b7e-86ce-41d792dd8a08 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.652250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b88fe-4081-4b7e-86ce-41d792dd8a08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.654406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b88fe-4081-4b7e-86ce-41d792dd8a08 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.660644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.663968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 387/776] 10.222.0.78 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.665068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b88fe-4081-4b7e-86ce-41d792dd8a08 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.768877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06efe8c-3ad6-4633-928d-e2281456923e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.813506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e9c5b-10a1-4600-8df0-be6b41702a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.813893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b88fe-4081-4b7e-86ce-41d792dd8a08 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.814714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e9c5b-10a1-4600-8df0-be6b41702a71 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.815213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e9c5b-10a1-4600-8df0-be6b41702a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.815885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e9c5b-10a1-4600-8df0-be6b41702a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.816633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e9c5b-10a1-4600-8df0-be6b41702a71 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=308bbd2bc1224533b573c80f0565252a) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:00.816943 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e9c5b-10a1-4600-8df0-be6b41702a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.819690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e9c5b-10a1-4600-8df0-be6b41702a71 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.826677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 390/777] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/role_assignments?user.id=308bbd2bc1224533b573c80f0565252a&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.829222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d3984-1b64-410b-af1e-9edb24776993 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.829756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d3984-1b64-410b-af1e-9edb24776993 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.830044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d3984-1b64-410b-af1e-9edb24776993 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.831855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 391/778] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:00.837018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd955a-b99f-4666-951c-966bbbd77b48 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.837877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd955a-b99f-4666-951c-966bbbd77b48 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.838256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd955a-b99f-4666-951c-966bbbd77b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.862871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f619f-b5e8-4ba8-a090-1a04c36c9c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.863810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f619f-b5e8-4ba8-a090-1a04c36c9c89 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.864216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f619f-b5e8-4ba8-a090-1a04c36c9c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.864638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f619f-b5e8-4ba8-a090-1a04c36c9c89 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.864901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f4fd955a-b99f-4666-951c-966bbbd77b48 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:00.865567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f619f-b5e8-4ba8-a090-1a04c36c9c89 None admin] RBAC: Adding query filter params (type=network) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:00.865969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f619f-b5e8-4ba8-a090-1a04c36c9c89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.866690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f4fd955a-b99f-4666-951c-966bbbd77b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:00.867608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f619f-b5e8-4ba8-a090-1a04c36c9c89 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.879580 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 388/779] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.881987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f619f-b5e8-4ba8-a090-1a04c36c9c89 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.898152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.958847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 392/780] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:01.020925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f619f-b5e8-4ba8-a090-1a04c36c9c89 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.055918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fb49a-7a99-48a0-b858-fccc265946d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.056534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fb49a-7a99-48a0-b858-fccc265946d7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.056795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fb49a-7a99-48a0-b858-fccc265946d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.057066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fb49a-7a99-48a0-b858-fccc265946d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.057811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fb49a-7a99-48a0-b858-fccc265946d7 None admin] RBAC: Adding query filter params (interface=public, service_id=e25a4811f0614f72858d155d8bfc2035, region_id=RegionOne) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:01.058119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fb49a-7a99-48a0-b858-fccc265946d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.059482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fb49a-7a99-48a0-b858-fccc265946d7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.070005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 389/781] 10.222.0.78 () {56 vars in 1251 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/endpoints?service_id=e25a4811f0614f72858d155d8bfc2035&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:01.079231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd955a-b99f-4666-951c-966bbbd77b48 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:01.131062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:01.221044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd955a-b99f-4666-951c-966bbbd77b48 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.268063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431253d0-e1e0-4432-9397-267853d30e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.269121 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431253d0-e1e0-4432-9397-267853d30e14 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.269597 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431253d0-e1e0-4432-9397-267853d30e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.270087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431253d0-e1e0-4432-9397-267853d30e14 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.270895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431253d0-e1e0-4432-9397-267853d30e14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.272589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431253d0-e1e0-4432-9397-267853d30e14 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.281906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 393/782] 10.222.0.78 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:01.300951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fb49a-7a99-48a0-b858-fccc265946d7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:01.395718 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:01.436039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fb49a-7a99-48a0-b858-fccc265946d7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.478649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28168c-3447-4181-b25e-5820250c6b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.479282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28168c-3447-4181-b25e-5820250c6b16 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.479559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28168c-3447-4181-b25e-5820250c6b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.479830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28168c-3447-4181-b25e-5820250c6b16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.496800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28168c-3447-4181-b25e-5820250c6b16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-5e28168c-3447-4181-b25e-5820250c6b16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:01.506787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:01.507770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:01.508615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 390/783] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:01.514122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431253d0-e1e0-4432-9397-267853d30e14 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:01.669600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:01.676385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431253d0-e1e0-4432-9397-267853d30e14 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.712448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3c09d-a645-40ac-8348-25ddc2ddb6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.713343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3c09d-a645-40ac-8348-25ddc2ddb6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.713766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3c09d-a645-40ac-8348-25ddc2ddb6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.714213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3c09d-a645-40ac-8348-25ddc2ddb6c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.715014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3c09d-a645-40ac-8348-25ddc2ddb6c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:01.715459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3c09d-a645-40ac-8348-25ddc2ddb6c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.716958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3c09d-a645-40ac-8348-25ddc2ddb6c1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.726455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 394/784] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:01.732200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28168c-3447-4181-b25e-5820250c6b16 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:01.870592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28168c-3447-4181-b25e-5820250c6b16 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.906929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:01.914559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ace0e1-7357-4167-afbd-c6cd50c50b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.915201 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ace0e1-7357-4167-afbd-c6cd50c50b6d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.915813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ace0e1-7357-4167-afbd-c6cd50c50b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.917467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ace0e1-7357-4167-afbd-c6cd50c50b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.929651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ace0e1-7357-4167-afbd-c6cd50c50b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.931052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ace0e1-7357-4167-afbd-c6cd50c50b6d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c3ace0e1-7357-4167-afbd-c6cd50c50b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:01.951650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:01.953303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:01.954218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:01.954218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:01.954218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:01.954218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 391/785] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:01.960532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3c09d-a645-40ac-8348-25ddc2ddb6c1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.115813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3c09d-a645-40ac-8348-25ddc2ddb6c1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:02.159458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1eb83-70ac-40c4-b80f-07eec7436617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:02.160077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1eb83-70ac-40c4-b80f-07eec7436617 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.160358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1eb83-70ac-40c4-b80f-07eec7436617 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.160635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1eb83-70ac-40c4-b80f-07eec7436617 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.161305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1eb83-70ac-40c4-b80f-07eec7436617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:02.161609 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1eb83-70ac-40c4-b80f-07eec7436617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:02.162976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1eb83-70ac-40c4-b80f-07eec7436617 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:02.181577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 395/786] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:02.190232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ace0e1-7357-4167-afbd-c6cd50c50b6d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.410123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fa6df-f948-4a6e-b3c1-2f0aa24680b3 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.410482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fa6df-f948-4a6e-b3c1-2f0aa24680b3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.410778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fa6df-f948-4a6e-b3c1-2f0aa24680b3 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.412164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 396/787] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:02.419481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73562eac-57fb-4a30-b982-263757490d00 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.419757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73562eac-57fb-4a30-b982-263757490d00 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.420032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73562eac-57fb-4a30-b982-263757490d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.453180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-73562eac-57fb-4a30-b982-263757490d00 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:02.454303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-73562eac-57fb-4a30-b982-263757490d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:02.456850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ace0e1-7357-4167-afbd-c6cd50c50b6d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:02.485400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:02.495209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd1436-1980-489d-b9e4-8f0473415012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:02.496270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd1436-1980-489d-b9e4-8f0473415012 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.496574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd1436-1980-489d-b9e4-8f0473415012 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.496936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd1436-1980-489d-b9e4-8f0473415012 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.504663 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd1436-1980-489d-b9e4-8f0473415012 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:02.506250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd1436-1980-489d-b9e4-8f0473415012 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-8cfd1436-1980-489d-b9e4-8f0473415012 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:02.522627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:02.523566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:02.524640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 392/788] 10.222.0.78 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:02.531421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd1436-1980-489d-b9e4-8f0473415012 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.613637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 397/789] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:02.617489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:02.694050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd1436-1980-489d-b9e4-8f0473415012 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:02.752594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73ee7e-566b-43e6-bd0e-a1ca58bcf94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:02.753642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73ee7e-566b-43e6-bd0e-a1ca58bcf94a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.754215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73ee7e-566b-43e6-bd0e-a1ca58bcf94a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.754706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73ee7e-566b-43e6-bd0e-a1ca58bcf94a None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.755640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73ee7e-566b-43e6-bd0e-a1ca58bcf94a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:02.756136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73ee7e-566b-43e6-bd0e-a1ca58bcf94a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:02.757779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73ee7e-566b-43e6-bd0e-a1ca58bcf94a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:02.787332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 393/790] 10.222.0.78 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:02.792636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73562eac-57fb-4a30-b982-263757490d00 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.945445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73562eac-57fb-4a30-b982-263757490d00 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:02.997954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b62d8-a3c2-4a0e-a837-837e8236ea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:02.999403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b62d8-a3c2-4a0e-a837-837e8236ea0d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.999907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b62d8-a3c2-4a0e-a837-837e8236ea0d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.000367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b62d8-a3c2-4a0e-a837-837e8236ea0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.014150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73ee7e-566b-43e6-bd0e-a1ca58bcf94a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.040107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.144649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b62d8-a3c2-4a0e-a837-837e8236ea0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.146177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b62d8-a3c2-4a0e-a837-837e8236ea0d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.188626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73ee7e-566b-43e6-bd0e-a1ca58bcf94a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.191091 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.235305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb0855-9892-44bd-a73a-6176cafd7a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.236547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb0855-9892-44bd-a73a-6176cafd7a2d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.237033 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb0855-9892-44bd-a73a-6176cafd7a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.238086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb0855-9892-44bd-a73a-6176cafd7a2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.263899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 398/791] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:03.264802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb0855-9892-44bd-a73a-6176cafd7a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:03.265738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b25e7-1b46-403b-a2fb-d2f363c5c775 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.266113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b25e7-1b46-403b-a2fb-d2f363c5c775 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.266704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b25e7-1b46-403b-a2fb-d2f363c5c775 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.268446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 399/792] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9ddb0855-9892-44bd-a73a-6176cafd7a2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:03.271708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:03.272752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:03.273586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:03.274198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f1278-9256-410a-b07d-e97c80800692 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.274198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f1278-9256-410a-b07d-e97c80800692 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.274198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f1278-9256-410a-b07d-e97c80800692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.276793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 394/793] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:03.278087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb0855-9892-44bd-a73a-6176cafd7a2d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.304390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b21f1278-9256-410a-b07d-e97c80800692 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:03.305891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b21f1278-9256-410a-b07d-e97c80800692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:03.356505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.383789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 400/794] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:03.384955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f1278-9256-410a-b07d-e97c80800692 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.397843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb0855-9892-44bd-a73a-6176cafd7a2d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.438428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6556e-60b5-4000-91a1-178f65895f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.439242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6556e-60b5-4000-91a1-178f65895f81 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.439591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6556e-60b5-4000-91a1-178f65895f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.439925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6556e-60b5-4000-91a1-178f65895f81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.451504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6556e-60b5-4000-91a1-178f65895f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.453916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6556e-60b5-4000-91a1-178f65895f81 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-72c6556e-60b5-4000-91a1-178f65895f81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:03.470170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:03.471151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:03.472039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:03.472039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:03.472039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:03.476021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 395/795] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:03.477645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6f9cf-d5f5-4320-ad29-947b65f08109 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.477994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6f9cf-d5f5-4320-ad29-947b65f08109 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.478374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6f9cf-d5f5-4320-ad29-947b65f08109 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.480035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 396/796] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:03.480918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6f9cf-d5f5-4320-ad29-947b65f08109 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.499177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.501687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f1278-9256-410a-b07d-e97c80800692 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.540988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf84c5-f820-4212-b314-2b745859d18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.542053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf84c5-f820-4212-b314-2b745859d18d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.542579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf84c5-f820-4212-b314-2b745859d18d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.543021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf84c5-f820-4212-b314-2b745859d18d None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.543917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf84c5-f820-4212-b314-2b745859d18d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:03.544435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf84c5-f820-4212-b314-2b745859d18d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.546673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf84c5-f820-4212-b314-2b745859d18d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.557324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 401/797] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:03.563285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4dadf-553d-465a-b700-e03afc120599 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.563824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4dadf-553d-465a-b700-e03afc120599 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.564300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4dadf-553d-465a-b700-e03afc120599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.583899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1ac4dadf-553d-465a-b700-e03afc120599 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:03.585490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1ac4dadf-553d-465a-b700-e03afc120599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:03.593948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6f9cf-d5f5-4320-ad29-947b65f08109 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.625058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.631538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01506e9-3513-4526-96bc-05e103bc1301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.632150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01506e9-3513-4526-96bc-05e103bc1301 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.632416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01506e9-3513-4526-96bc-05e103bc1301 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.632685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01506e9-3513-4526-96bc-05e103bc1301 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.633372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01506e9-3513-4526-96bc-05e103bc1301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:03.633729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01506e9-3513-4526-96bc-05e103bc1301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.635918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01506e9-3513-4526-96bc-05e103bc1301 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.653271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 397/798] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:03.654159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01506e9-3513-4526-96bc-05e103bc1301 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.667226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 402/799] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:03.669103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4dadf-553d-465a-b700-e03afc120599 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.744549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.771127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01506e9-3513-4526-96bc-05e103bc1301 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.798784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4dadf-553d-465a-b700-e03afc120599 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.805456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce4da3-e85f-4727-bc53-6b17701bec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.806080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce4da3-e85f-4727-bc53-6b17701bec8b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.806357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce4da3-e85f-4727-bc53-6b17701bec8b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.806624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce4da3-e85f-4727-bc53-6b17701bec8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.817740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce4da3-e85f-4727-bc53-6b17701bec8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.819543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce4da3-e85f-4727-bc53-6b17701bec8b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-67ce4da3-e85f-4727-bc53-6b17701bec8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:03.834829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:03.835995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:03.837109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:03.837109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:03.837109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:03.840752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 398/800] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:03.841614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce4da3-e85f-4727-bc53-6b17701bec8b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.842341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aad95f-1975-4ffa-9794-231c0c9b67cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.843481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aad95f-1975-4ffa-9794-231c0c9b67cc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.843908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aad95f-1975-4ffa-9794-231c0c9b67cc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.844325 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aad95f-1975-4ffa-9794-231c0c9b67cc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.845156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aad95f-1975-4ffa-9794-231c0c9b67cc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.846912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aad95f-1975-4ffa-9794-231c0c9b67cc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-12aad95f-1975-4ffa-9794-231c0c9b67cc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:03.856781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:03.858065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:03.859552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:03.862070 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 403/801] 10.222.0.78 () {56 vars in 1129 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:03.863025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aad95f-1975-4ffa-9794-231c0c9b67cc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.898433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.949835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce4da3-e85f-4727-bc53-6b17701bec8b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.970644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aad95f-1975-4ffa-9794-231c0c9b67cc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.975210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.984348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c46d64-dd30-4540-9c8e-8683da9211a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.985010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c46d64-dd30-4540-9c8e-8683da9211a5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.985316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c46d64-dd30-4540-9c8e-8683da9211a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.985583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c46d64-dd30-4540-9c8e-8683da9211a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.992192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c46d64-dd30-4540-9c8e-8683da9211a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.993711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c46d64-dd30-4540-9c8e-8683da9211a5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.004631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cc54f-0287-49cd-9ed6-83b24909f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-42c46d64-dd30-4540-9c8e-8683da9211a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:04.005119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:04.006012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:04.006879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:04.006879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:04.006879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:04.006879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:04.006879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:04.006879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:04.006879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:04.006879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:04.007602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cc54f-0287-49cd-9ed6-83b24909f457 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.007602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cc54f-0287-49cd-9ed6-83b24909f457 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.007602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cc54f-0287-49cd-9ed6-83b24909f457 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.007783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 399/802] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:04.007858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cc54f-0287-49cd-9ed6-83b24909f457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:04.007858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cc54f-0287-49cd-9ed6-83b24909f457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.007977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb888f-7b29-4d3d-816f-9d3a47f5ef03 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.007977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb888f-7b29-4d3d-816f-9d3a47f5ef03 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.007977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb888f-7b29-4d3d-816f-9d3a47f5ef03 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.008160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cc54f-0287-49cd-9ed6-83b24909f457 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.012956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 400/803] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:04.012956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb888f-7b29-4d3d-816f-9d3a47f5ef03 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.024496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 404/804] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.025343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cc54f-0287-49cd-9ed6-83b24909f457 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.113428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb888f-7b29-4d3d-816f-9d3a47f5ef03 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.121440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.128479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cc54f-0287-49cd-9ed6-83b24909f457 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.146603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779250da-bffc-4265-ad1a-71cd472ff414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.147199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779250da-bffc-4265-ad1a-71cd472ff414 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.147476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779250da-bffc-4265-ad1a-71cd472ff414 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.147782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779250da-bffc-4265-ad1a-71cd472ff414 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.148412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779250da-bffc-4265-ad1a-71cd472ff414 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:04.148710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779250da-bffc-4265-ad1a-71cd472ff414 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.150082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779250da-bffc-4265-ad1a-71cd472ff414 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.154500 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 401/805] 10.222.0.78 () {56 vars in 1124 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.155272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779250da-bffc-4265-ad1a-71cd472ff414 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.162274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf7c12-af1c-487c-b280-e46ce89f1deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.162896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf7c12-af1c-487c-b280-e46ce89f1deb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.163170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf7c12-af1c-487c-b280-e46ce89f1deb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.163435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf7c12-af1c-487c-b280-e46ce89f1deb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.174755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf7c12-af1c-487c-b280-e46ce89f1deb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-febf7c12-af1c-487c-b280-e46ce89f1deb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:04.179470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:04.180456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:04.181305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 405/806] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:04.182029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f3a90-f17c-4a67-b2a6-04a1d58c9bc7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.182311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f3a90-f17c-4a67-b2a6-04a1d58c9bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.182579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f3a90-f17c-4a67-b2a6-04a1d58c9bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.201196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d77f3a90-f17c-4a67-b2a6-04a1d58c9bc7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:04.202399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d77f3a90-f17c-4a67-b2a6-04a1d58c9bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:04.212606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.264002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779250da-bffc-4265-ad1a-71cd472ff414 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.271136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 406/807] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:04.271975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f3a90-f17c-4a67-b2a6-04a1d58c9bc7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.301362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d71dbc-ba4e-4ef6-bcf8-f00cf02a3b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.301961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d71dbc-ba4e-4ef6-bcf8-f00cf02a3b31 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.302236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d71dbc-ba4e-4ef6-bcf8-f00cf02a3b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.302503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d71dbc-ba4e-4ef6-bcf8-f00cf02a3b31 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.303186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d71dbc-ba4e-4ef6-bcf8-f00cf02a3b31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:04.303491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d71dbc-ba4e-4ef6-bcf8-f00cf02a3b31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.305059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d71dbc-ba4e-4ef6-bcf8-f00cf02a3b31 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.318176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 402/808] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.318985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d71dbc-ba4e-4ef6-bcf8-f00cf02a3b31 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.378958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.396372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f3a90-f17c-4a67-b2a6-04a1d58c9bc7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.426645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d71dbc-ba4e-4ef6-bcf8-f00cf02a3b31 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.432477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ef2d-11b5-4cec-871f-66f78d2ddd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.433124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ef2d-11b5-4cec-871f-66f78d2ddd5d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.433402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ef2d-11b5-4cec-871f-66f78d2ddd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.433674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ef2d-11b5-4cec-871f-66f78d2ddd5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.440203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.441456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08ef2d-11b5-4cec-871f-66f78d2ddd5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.443021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08ef2d-11b5-4cec-871f-66f78d2ddd5d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8b08ef2d-11b5-4cec-871f-66f78d2ddd5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:04.454797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:04.455748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 407/809] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:04.456712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ef2d-11b5-4cec-871f-66f78d2ddd5d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.473129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cc38-92ac-4bd4-876d-5cc33f94f9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.473745 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cc38-92ac-4bd4-876d-5cc33f94f9ea None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.474025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cc38-92ac-4bd4-876d-5cc33f94f9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.474292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cc38-92ac-4bd4-876d-5cc33f94f9ea None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.474926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463cc38-92ac-4bd4-876d-5cc33f94f9ea None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:04.475337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463cc38-92ac-4bd4-876d-5cc33f94f9ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.476873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463cc38-92ac-4bd4-876d-5cc33f94f9ea None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.487559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 403/810] 10.222.0.78 () {56 vars in 1124 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.488405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cc38-92ac-4bd4-876d-5cc33f94f9ea None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.562046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ef2d-11b5-4cec-871f-66f78d2ddd5d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.591441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cc38-92ac-4bd4-876d-5cc33f94f9ea None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.595680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e91fb-89b2-4a71-9141-b29d40023027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.596278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e91fb-89b2-4a71-9141-b29d40023027 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.596552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e91fb-89b2-4a71-9141-b29d40023027 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.596815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e91fb-89b2-4a71-9141-b29d40023027 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.597494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e91fb-89b2-4a71-9141-b29d40023027 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:04.597795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e91fb-89b2-4a71-9141-b29d40023027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.599179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e91fb-89b2-4a71-9141-b29d40023027 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.607505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 408/811] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.608283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e91fb-89b2-4a71-9141-b29d40023027 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.617418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.630583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f25b9b-5cec-44d9-9332-8d1427c40e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.631118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f25b9b-5cec-44d9-9332-8d1427c40e23 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.631392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f25b9b-5cec-44d9-9332-8d1427c40e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.631664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f25b9b-5cec-44d9-9332-8d1427c40e23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.632386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f25b9b-5cec-44d9-9332-8d1427c40e23 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=f9b88c495fcb47a289a6a45276c891e0) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:04.632696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f25b9b-5cec-44d9-9332-8d1427c40e23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.634141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f25b9b-5cec-44d9-9332-8d1427c40e23 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.640102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 404/812] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/role_assignments?user.id=f9b88c495fcb47a289a6a45276c891e0&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.640797 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f25b9b-5cec-44d9-9332-8d1427c40e23 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.687781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.731924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e91fb-89b2-4a71-9141-b29d40023027 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.762353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f25b9b-5cec-44d9-9332-8d1427c40e23 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.773212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3942b5-bc2b-4b10-8a4d-37ce76de70a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.774053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3942b5-bc2b-4b10-8a4d-37ce76de70a2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.774164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3942b5-bc2b-4b10-8a4d-37ce76de70a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.774374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3942b5-bc2b-4b10-8a4d-37ce76de70a2 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.775105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3942b5-bc2b-4b10-8a4d-37ce76de70a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:04.775411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3942b5-bc2b-4b10-8a4d-37ce76de70a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.776857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3942b5-bc2b-4b10-8a4d-37ce76de70a2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.799016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 409/813] 10.222.0.78 () {56 vars in 1135 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.799932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3942b5-bc2b-4b10-8a4d-37ce76de70a2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.807220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64de1-ae94-4eb7-98fd-aadacafed196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.807836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64de1-ae94-4eb7-98fd-aadacafed196 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.808113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64de1-ae94-4eb7-98fd-aadacafed196 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.808381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64de1-ae94-4eb7-98fd-aadacafed196 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.809265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb64de1-ae94-4eb7-98fd-aadacafed196 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.810760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb64de1-ae94-4eb7-98fd-aadacafed196 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.835066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 405/814] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:04.835278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64de1-ae94-4eb7-98fd-aadacafed196 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.881681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.927084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3942b5-bc2b-4b10-8a4d-37ce76de70a2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.975406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dae21-b9a7-4a90-83bb-caefaf598cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.976054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dae21-b9a7-4a90-83bb-caefaf598cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.976334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dae21-b9a7-4a90-83bb-caefaf598cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.976600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dae21-b9a7-4a90-83bb-caefaf598cd7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.977226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dae21-b9a7-4a90-83bb-caefaf598cd7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.979320 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dae21-b9a7-4a90-83bb-caefaf598cd7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.984955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3a3dae21-b9a7-4a90-83bb-caefaf598cd7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:04.994238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:04.995422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:04.996400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 410/815] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:04.997539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dae21-b9a7-4a90-83bb-caefaf598cd7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.998461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64de1-ae94-4eb7-98fd-aadacafed196 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.047637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacaf31-ccec-4769-95f8-ada47754b500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.048273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacaf31-ccec-4769-95f8-ada47754b500 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.048792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacaf31-ccec-4769-95f8-ada47754b500 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.048792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacaf31-ccec-4769-95f8-ada47754b500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.068569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacaf31-ccec-4769-95f8-ada47754b500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.070583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacaf31-ccec-4769-95f8-ada47754b500 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-6dacaf31-ccec-4769-95f8-ada47754b500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:05.091497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:05.092506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:05.093427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:05.093427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:05.093427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:05.093427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 406/816] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:05.094125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacaf31-ccec-4769-95f8-ada47754b500 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.132358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dae21-b9a7-4a90-83bb-caefaf598cd7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.157503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.169271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9abc3-98ee-4332-af7f-458e87aa56bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.170020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9abc3-98ee-4332-af7f-458e87aa56bd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.170356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9abc3-98ee-4332-af7f-458e87aa56bd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.170719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9abc3-98ee-4332-af7f-458e87aa56bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.183308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9abc3-98ee-4332-af7f-458e87aa56bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.184958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9abc3-98ee-4332-af7f-458e87aa56bd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-50f9abc3-98ee-4332-af7f-458e87aa56bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:05.201052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:05.202237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:05.203718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:05.203718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:05.203718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:05.208504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 411/817] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:05.209386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9abc3-98ee-4332-af7f-458e87aa56bd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.219686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacaf31-ccec-4769-95f8-ada47754b500 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.269970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bad796-65aa-4110-95e8-d3bdc05593dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.271498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bad796-65aa-4110-95e8-d3bdc05593dd None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.289365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bad796-65aa-4110-95e8-d3bdc05593dd None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.289365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bad796-65aa-4110-95e8-d3bdc05593dd None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.289365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bad796-65aa-4110-95e8-d3bdc05593dd None admin] RBAC: Adding query filter params (name=network) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:05.289365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bad796-65aa-4110-95e8-d3bdc05593dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.289365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bad796-65aa-4110-95e8-d3bdc05593dd None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.289365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 407/818] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:05.290311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bad796-65aa-4110-95e8-d3bdc05593dd None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.300674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.330647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9abc3-98ee-4332-af7f-458e87aa56bd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.366201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7a252-c8da-4344-9f18-bc55eac30474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.367230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7a252-c8da-4344-9f18-bc55eac30474 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.367734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7a252-c8da-4344-9f18-bc55eac30474 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.368198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7a252-c8da-4344-9f18-bc55eac30474 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.369023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7a252-c8da-4344-9f18-bc55eac30474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:05.369559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7a252-c8da-4344-9f18-bc55eac30474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.371112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7a252-c8da-4344-9f18-bc55eac30474 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.388999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 412/819] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:05.390262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7a252-c8da-4344-9f18-bc55eac30474 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.420995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bad796-65aa-4110-95e8-d3bdc05593dd None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.431688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.457917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888cc8-6dcb-4cc7-999f-26f6913151e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.458933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888cc8-6dcb-4cc7-999f-26f6913151e4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.459213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888cc8-6dcb-4cc7-999f-26f6913151e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.459479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888cc8-6dcb-4cc7-999f-26f6913151e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.460253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888cc8-6dcb-4cc7-999f-26f6913151e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:05.460693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888cc8-6dcb-4cc7-999f-26f6913151e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.462634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888cc8-6dcb-4cc7-999f-26f6913151e4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.484554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 408/820] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:05.484554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888cc8-6dcb-4cc7-999f-26f6913151e4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.509957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7a252-c8da-4344-9f18-bc55eac30474 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.546187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956499bc-be3b-47e0-a840-2c67a269ef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.547677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956499bc-be3b-47e0-a840-2c67a269ef6a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.548152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956499bc-be3b-47e0-a840-2c67a269ef6a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.548706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956499bc-be3b-47e0-a840-2c67a269ef6a None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.549651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956499bc-be3b-47e0-a840-2c67a269ef6a None admin] RBAC: Adding query filter params (type=network) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:05.550224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956499bc-be3b-47e0-a840-2c67a269ef6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.552024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956499bc-be3b-47e0-a840-2c67a269ef6a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.561560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 413/821] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:05.563690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956499bc-be3b-47e0-a840-2c67a269ef6a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.572957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.613694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888cc8-6dcb-4cc7-999f-26f6913151e4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.650324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1bb0b-5575-4e89-911c-2a23c2f33197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.651445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1bb0b-5575-4e89-911c-2a23c2f33197 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.651926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1bb0b-5575-4e89-911c-2a23c2f33197 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.652399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1bb0b-5575-4e89-911c-2a23c2f33197 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.661231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1bb0b-5575-4e89-911c-2a23c2f33197 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.663252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1bb0b-5575-4e89-911c-2a23c2f33197 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a6d1bb0b-5575-4e89-911c-2a23c2f33197 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:05.678813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:05.679779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:05.680699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:05.687401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 409/822] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:05.687401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1bb0b-5575-4e89-911c-2a23c2f33197 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.693701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.696848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956499bc-be3b-47e0-a840-2c67a269ef6a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.745230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e76783-6992-4055-9b92-5a362411307d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.746420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e76783-6992-4055-9b92-5a362411307d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.747220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e76783-6992-4055-9b92-5a362411307d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.747738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e76783-6992-4055-9b92-5a362411307d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.761637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e76783-6992-4055-9b92-5a362411307d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.763400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e76783-6992-4055-9b92-5a362411307d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-98e76783-6992-4055-9b92-5a362411307d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:05.781016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:05.781957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:05.782832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:05.782832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:05.782832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:05.782832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:05.782832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:05.782832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:05.782832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:05.782832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:05.790368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 414/823] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:05.790368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e76783-6992-4055-9b92-5a362411307d None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.808582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1bb0b-5575-4e89-911c-2a23c2f33197 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.842617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.850111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22b8e9-dae5-4317-a140-eac9cab5b6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.851367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22b8e9-dae5-4317-a140-eac9cab5b6c8 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.852178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22b8e9-dae5-4317-a140-eac9cab5b6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.852550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22b8e9-dae5-4317-a140-eac9cab5b6c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.853673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22b8e9-dae5-4317-a140-eac9cab5b6c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.855074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22b8e9-dae5-4317-a140-eac9cab5b6c8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.889105 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 410/824] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:05 2022] POST /identity/v3/endpoints => generated 337 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:05.889105 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22b8e9-dae5-4317-a140-eac9cab5b6c8 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.951503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e76783-6992-4055-9b92-5a362411307d None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.025384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22b8e9-dae5-4317-a140-eac9cab5b6c8 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.034522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4d9ec-3c09-43f7-adaf-1e2e0b94f313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.035386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4d9ec-3c09-43f7-adaf-1e2e0b94f313 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.035948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4d9ec-3c09-43f7-adaf-1e2e0b94f313 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.036467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4d9ec-3c09-43f7-adaf-1e2e0b94f313 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.037437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4d9ec-3c09-43f7-adaf-1e2e0b94f313 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:06.037938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4d9ec-3c09-43f7-adaf-1e2e0b94f313 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.039699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4d9ec-3c09-43f7-adaf-1e2e0b94f313 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.071634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309fe8-ab55-4b8e-92ea-17b77b121a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.072361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309fe8-ab55-4b8e-92ea-17b77b121a52 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.072647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309fe8-ab55-4b8e-92ea-17b77b121a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.072923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309fe8-ab55-4b8e-92ea-17b77b121a52 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.073689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309fe8-ab55-4b8e-92ea-17b77b121a52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:06.073999 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309fe8-ab55-4b8e-92ea-17b77b121a52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.075438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309fe8-ab55-4b8e-92ea-17b77b121a52 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.086779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 415/825] 10.222.0.78 () {56 vars in 1149 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:06.091642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 411/826] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:06.098127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4d9ec-3c09-43f7-adaf-1e2e0b94f313 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.098385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309fe8-ab55-4b8e-92ea-17b77b121a52 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.135114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:06.174391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:06.242073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4d9ec-3c09-43f7-adaf-1e2e0b94f313 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.248914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309fe8-ab55-4b8e-92ea-17b77b121a52 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.289843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e52104-365f-427b-ad42-b4794e198860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.290389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52104-365f-427b-ad42-b4794e198860 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.290669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52104-365f-427b-ad42-b4794e198860 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.291285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52104-365f-427b-ad42-b4794e198860 None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/c53b9bcd675940c19febbc8fefb1f28b/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.293764 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e52104-365f-427b-ad42-b4794e198860 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=c53b9bcd675940c19febbc8fefb1f28b, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.295920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e52104-365f-427b-ad42-b4794e198860 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.303960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88766d-fc84-41c6-bafd-bcb4c4f4ff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.305007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88766d-fc84-41c6-bafd-bcb4c4f4ff54 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.305528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88766d-fc84-41c6-bafd-bcb4c4f4ff54 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.306024 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88766d-fc84-41c6-bafd-bcb4c4f4ff54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.321185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 416/827] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:06 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/c53b9bcd675940c19febbc8fefb1f28b/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:06.326150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88766d-fc84-41c6-bafd-bcb4c4f4ff54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.328687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88766d-fc84-41c6-bafd-bcb4c4f4ff54 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ac88766d-fc84-41c6-bafd-bcb4c4f4ff54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:06.347459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:06.348505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:06.349436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:06.349436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:06.349436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:06.352030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 412/828] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:06.357564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e52104-365f-427b-ad42-b4794e198860 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.454127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:06.481434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e52104-365f-427b-ad42-b4794e198860 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.524585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416151bb-8c93-48f5-865d-a5db17d1a29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.525577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416151bb-8c93-48f5-865d-a5db17d1a29e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.526078 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416151bb-8c93-48f5-865d-a5db17d1a29e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.526545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416151bb-8c93-48f5-865d-a5db17d1a29e None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.527386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416151bb-8c93-48f5-865d-a5db17d1a29e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:06.527884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416151bb-8c93-48f5-865d-a5db17d1a29e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.529549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416151bb-8c93-48f5-865d-a5db17d1a29e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.543678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 417/829] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:06.550339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88766d-fc84-41c6-bafd-bcb4c4f4ff54 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.650484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:06.659927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88766d-fc84-41c6-bafd-bcb4c4f4ff54 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.694828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a89181-892d-4979-88e5-ad38aad39899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.695451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a89181-892d-4979-88e5-ad38aad39899 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.695727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a89181-892d-4979-88e5-ad38aad39899 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.695996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a89181-892d-4979-88e5-ad38aad39899 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.697952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17413cfd-562b-4efe-b8d0-418d2217f0c1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.698449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17413cfd-562b-4efe-b8d0-418d2217f0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.699007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17413cfd-562b-4efe-b8d0-418d2217f0c1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.700924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 418/830] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:06.703975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a89181-892d-4979-88e5-ad38aad39899 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.705442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a89181-892d-4979-88e5-ad38aad39899 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.706618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437a632-481e-4415-b79d-0f2b56a7e5d8 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.707428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437a632-481e-4415-b79d-0f2b56a7e5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.708366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437a632-481e-4415-b79d-0f2b56a7e5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-63a89181-892d-4979-88e5-ad38aad39899 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:06.717396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:06.718956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:06.720236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:06.720236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:06.720236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:06.720236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:06.720236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:06.720236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:06.720236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:06.720236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:06.720236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 413/831] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:06.724885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a89181-892d-4979-88e5-ad38aad39899 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.733051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3437a632-481e-4415-b79d-0f2b56a7e5d8 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:06.735661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3437a632-481e-4415-b79d-0f2b56a7e5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:06.834102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 419/832] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:06.835815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a89181-892d-4979-88e5-ad38aad39899 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.878097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174994-adb8-4745-914a-77a1975d40f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.878758 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174994-adb8-4745-914a-77a1975d40f2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.879039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174994-adb8-4745-914a-77a1975d40f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.879370 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174994-adb8-4745-914a-77a1975d40f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.880116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41174994-adb8-4745-914a-77a1975d40f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:06.880421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41174994-adb8-4745-914a-77a1975d40f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.881912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41174994-adb8-4745-914a-77a1975d40f2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.895389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 414/833] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:06.901302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437a632-481e-4415-b79d-0f2b56a7e5d8 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.970388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.009655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437a632-481e-4415-b79d-0f2b56a7e5d8 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.046073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6687a9-919e-4e49-8c54-e4ce2463ccfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.047628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6687a9-919e-4e49-8c54-e4ce2463ccfb None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.048135 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6687a9-919e-4e49-8c54-e4ce2463ccfb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.048604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6687a9-919e-4e49-8c54-e4ce2463ccfb None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/308bbd2bc1224533b573c80f0565252a/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.053830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872864d-4dd3-495e-b951-f29ca2b70cfa None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.054222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872864d-4dd3-495e-b951-f29ca2b70cfa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.054464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872864d-4dd3-495e-b951-f29ca2b70cfa None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.056019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 415/834] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:07.061086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fad468-507a-4980-8d1b-78e3d9ccbc63 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.061437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fad468-507a-4980-8d1b-78e3d9ccbc63 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.061742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fad468-507a-4980-8d1b-78e3d9ccbc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.067058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6687a9-919e-4e49-8c54-e4ce2463ccfb None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=308bbd2bc1224533b573c80f0565252a, role_id=fd7dbf5fe1ae4cdf8a10f67c6921d3da)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.068773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6687a9-919e-4e49-8c54-e4ce2463ccfb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.083688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-07fad468-507a-4980-8d1b-78e3d9ccbc63 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:07.085056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-07fad468-507a-4980-8d1b-78e3d9ccbc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:07.087016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 420/835] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:06 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/308bbd2bc1224533b573c80f0565252a/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:07.103268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6687a9-919e-4e49-8c54-e4ce2463ccfb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.103931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.188901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 416/836] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:07.219091 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6687a9-919e-4e49-8c54-e4ce2463ccfb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.238249 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.259305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d78ab-2d96-4076-bc89-b9dddca4bb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.260210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d78ab-2d96-4076-bc89-b9dddca4bb17 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.260847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d78ab-2d96-4076-bc89-b9dddca4bb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.263354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d78ab-2d96-4076-bc89-b9dddca4bb17 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.263354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d78ab-2d96-4076-bc89-b9dddca4bb17 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.264231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d78ab-2d96-4076-bc89-b9dddca4bb17 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4c6d78ab-2d96-4076-bc89-b9dddca4bb17 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:07.277582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:07.278797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:07.279940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 421/837] 10.222.0.78 () {56 vars in 1132 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:07.285746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fad468-507a-4980-8d1b-78e3d9ccbc63 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.398542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fad468-507a-4980-8d1b-78e3d9ccbc63 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.434854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aca1f3-2c3d-4c5d-abde-ed84754e21b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.435808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aca1f3-2c3d-4c5d-abde-ed84754e21b9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.436288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aca1f3-2c3d-4c5d-abde-ed84754e21b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.436877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aca1f3-2c3d-4c5d-abde-ed84754e21b9 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.437918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aca1f3-2c3d-4c5d-abde-ed84754e21b9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:07.438422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aca1f3-2c3d-4c5d-abde-ed84754e21b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.439969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aca1f3-2c3d-4c5d-abde-ed84754e21b9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.445760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 417/838] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:07.454281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d78ab-2d96-4076-bc89-b9dddca4bb17 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.503466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aca1f3-2c3d-4c5d-abde-ed84754e21b9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.517575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.571157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d78ab-2d96-4076-bc89-b9dddca4bb17 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.610199 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a746f1-ad14-476f-b70d-99ea3d7c9912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.610815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a746f1-ad14-476f-b70d-99ea3d7c9912 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.611097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a746f1-ad14-476f-b70d-99ea3d7c9912 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.611392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a746f1-ad14-476f-b70d-99ea3d7c9912 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.611988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a746f1-ad14-476f-b70d-99ea3d7c9912 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:07.612291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a746f1-ad14-476f-b70d-99ea3d7c9912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.613802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a746f1-ad14-476f-b70d-99ea3d7c9912 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.618981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 422/839] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:07.630938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aca1f3-2c3d-4c5d-abde-ed84754e21b9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.671580 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d7f8a-8d78-4e3a-88bc-968db3d9feda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.672669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d7f8a-8d78-4e3a-88bc-968db3d9feda None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.673204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d7f8a-8d78-4e3a-88bc-968db3d9feda None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.673724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d7f8a-8d78-4e3a-88bc-968db3d9feda None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.674539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d7f8a-8d78-4e3a-88bc-968db3d9feda None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.676160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d7f8a-8d78-4e3a-88bc-968db3d9feda None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ec7d7f8a-8d78-4e3a-88bc-968db3d9feda None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:07.686561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:07.688170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:07.689002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:07.697113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 418/840] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:07.698844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a746f1-ad14-476f-b70d-99ea3d7c9912 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.752287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.812162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a746f1-ad14-476f-b70d-99ea3d7c9912 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.848255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005a271-5628-4720-b85a-3fa062d1bf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.849238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005a271-5628-4720-b85a-3fa062d1bf5e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.849720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005a271-5628-4720-b85a-3fa062d1bf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.850190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005a271-5628-4720-b85a-3fa062d1bf5e None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.851046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005a271-5628-4720-b85a-3fa062d1bf5e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:07.851546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005a271-5628-4720-b85a-3fa062d1bf5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.853128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005a271-5628-4720-b85a-3fa062d1bf5e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.863162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 423/841] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:07.864792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d7f8a-8d78-4e3a-88bc-968db3d9feda None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.938892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.970798 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d7f8a-8d78-4e3a-88bc-968db3d9feda None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:08.005663 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab25f3-fe55-4328-98ae-8e820426fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:08.006269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab25f3-fe55-4328-98ae-8e820426fc2e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.006547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab25f3-fe55-4328-98ae-8e820426fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.006815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab25f3-fe55-4328-98ae-8e820426fc2e None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.007450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab25f3-fe55-4328-98ae-8e820426fc2e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:08.007752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab25f3-fe55-4328-98ae-8e820426fc2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:08.009213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab25f3-fe55-4328-98ae-8e820426fc2e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:08.018843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 419/842] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:08.365033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49e615-a49d-48a7-9b0d-583acba8e2d4 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.365605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49e615-a49d-48a7-9b0d-583acba8e2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.365605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49e615-a49d-48a7-9b0d-583acba8e2d4 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.367169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 424/843] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:08.372296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5a790-32e5-4d64-9b69-557b5cf5d9b5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.372726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5a790-32e5-4d64-9b69-557b5cf5d9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.372898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5a790-32e5-4d64-9b69-557b5cf5d9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.391878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cdc5a790-32e5-4d64-9b69-557b5cf5d9b5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:08.393043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cdc5a790-32e5-4d64-9b69-557b5cf5d9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:08.468878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 420/844] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:08.744951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49e615-a49d-48a7-9b0d-583acba8e2d4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.855266 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49e615-a49d-48a7-9b0d-583acba8e2d4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:08.866228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:08.894602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174c65c-e8e3-4563-9d40-b6268a463e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:08.895242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174c65c-e8e3-4563-9d40-b6268a463e6f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.895520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174c65c-e8e3-4563-9d40-b6268a463e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.895790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174c65c-e8e3-4563-9d40-b6268a463e6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.906859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174c65c-e8e3-4563-9d40-b6268a463e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-9174c65c-e8e3-4563-9d40-b6268a463e6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:08.911795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:08.912705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:08.913561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 425/845] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:08.918846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5a790-32e5-4d64-9b69-557b5cf5d9b5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.992363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.031523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5a790-32e5-4d64-9b69-557b5cf5d9b5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.067630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51d671-b7c2-4651-9e4b-70e6cc45a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.068239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51d671-b7c2-4651-9e4b-70e6cc45a60f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.068522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51d671-b7c2-4651-9e4b-70e6cc45a60f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.068794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51d671-b7c2-4651-9e4b-70e6cc45a60f None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.069821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51d671-b7c2-4651-9e4b-70e6cc45a60f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:09.070130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51d671-b7c2-4651-9e4b-70e6cc45a60f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.071780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8dfa3-bd7b-448b-ab4b-8538791457d8 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.072063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8dfa3-bd7b-448b-ab4b-8538791457d8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.072334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8dfa3-bd7b-448b-ab4b-8538791457d8 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.074446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51d671-b7c2-4651-9e4b-70e6cc45a60f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.074558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 426/846] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:09.083383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cb737-e070-4878-bc8d-56ffc1e7b1d2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.083796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cb737-e070-4878-bc8d-56ffc1e7b1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.084192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cb737-e070-4878-bc8d-56ffc1e7b1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.089005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 421/847] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:09.095834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51d671-b7c2-4651-9e4b-70e6cc45a60f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.110226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8d9cb737-e070-4878-bc8d-56ffc1e7b1d2 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:09.112573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8d9cb737-e070-4878-bc8d-56ffc1e7b1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:09.207241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 427/848] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:09.211706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51d671-b7c2-4651-9e4b-70e6cc45a60f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.238433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.250873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91656f1-3479-42b2-b79a-bd9b877051f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.251598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91656f1-3479-42b2-b79a-bd9b877051f6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.251875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91656f1-3479-42b2-b79a-bd9b877051f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.252141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91656f1-3479-42b2-b79a-bd9b877051f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.264184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91656f1-3479-42b2-b79a-bd9b877051f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.266745 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91656f1-3479-42b2-b79a-bd9b877051f6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d91656f1-3479-42b2-b79a-bd9b877051f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:09.281334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:09.282309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:09.283220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:09.283220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:09.283220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:09.283220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 422/849] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:09.294337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cb737-e070-4878-bc8d-56ffc1e7b1d2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.418598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cb737-e070-4878-bc8d-56ffc1e7b1d2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.441431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.455693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33be9f-9efa-4898-9920-569697162a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.456306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33be9f-9efa-4898-9920-569697162a2b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.456584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33be9f-9efa-4898-9920-569697162a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.456855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33be9f-9efa-4898-9920-569697162a2b None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.457541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33be9f-9efa-4898-9920-569697162a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:09.457845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33be9f-9efa-4898-9920-569697162a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.459220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33be9f-9efa-4898-9920-569697162a2b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.471779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 428/850] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:09.477354 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91656f1-3479-42b2-b79a-bd9b877051f6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.485509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33be9f-9efa-4898-9920-569697162a2b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.590518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33be9f-9efa-4898-9920-569697162a2b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.596172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91656f1-3479-42b2-b79a-bd9b877051f6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.627026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45100d9d-d9db-40b8-afdf-65c1c7073262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.627655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45100d9d-d9db-40b8-afdf-65c1c7073262 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.627940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45100d9d-d9db-40b8-afdf-65c1c7073262 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.628213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45100d9d-d9db-40b8-afdf-65c1c7073262 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.632997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cde4c-5067-47ca-9e2f-f8263a14878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.633659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cde4c-5067-47ca-9e2f-f8263a14878d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.633931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cde4c-5067-47ca-9e2f-f8263a14878d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.634196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cde4c-5067-47ca-9e2f-f8263a14878d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.639648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45100d9d-d9db-40b8-afdf-65c1c7073262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:09.642524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cde4c-5067-47ca-9e2f-f8263a14878d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.643960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cde4c-5067-47ca-9e2f-f8263a14878d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-45100d9d-d9db-40b8-afdf-65c1c7073262 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:09.645359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:09.646278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:09.647138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 429/851] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:09.652413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45100d9d-d9db-40b8-afdf-65c1c7073262 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-119cde4c-5067-47ca-9e2f-f8263a14878d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:09.659420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:09.660877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:09.662612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:09.663253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 423/852] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:09.670285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cde4c-5067-47ca-9e2f-f8263a14878d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.681163 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.727124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.785299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45100d9d-d9db-40b8-afdf-65c1c7073262 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.789497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cde4c-5067-47ca-9e2f-f8263a14878d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.822468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f40eb-cec1-4986-bade-af6db387cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.823074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f40eb-cec1-4986-bade-af6db387cd62 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.823356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f40eb-cec1-4986-bade-af6db387cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.823627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f40eb-cec1-4986-bade-af6db387cd62 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.824287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f40eb-cec1-4986-bade-af6db387cd62 None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:09.824629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f40eb-cec1-4986-bade-af6db387cd62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.825980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f40eb-cec1-4986-bade-af6db387cd62 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.826145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78911197-754c-45e5-95cf-e9782287deb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.826699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78911197-754c-45e5-95cf-e9782287deb4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.826981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78911197-754c-45e5-95cf-e9782287deb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.827798 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78911197-754c-45e5-95cf-e9782287deb4 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.828633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78911197-754c-45e5-95cf-e9782287deb4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:09.828945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78911197-754c-45e5-95cf-e9782287deb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.830376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78911197-754c-45e5-95cf-e9782287deb4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.839897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 430/853] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:09.839897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835f66d-8062-4579-ae03-51f8598e9423 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.839897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835f66d-8062-4579-ae03-51f8598e9423 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.839897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835f66d-8062-4579-ae03-51f8598e9423 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.839897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 431/854] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:09.840717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835f66d-8062-4579-ae03-51f8598e9423 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.858940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 424/855] 10.222.0.78 () {56 vars in 1135 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:09.858940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2feb8-0264-4bac-b068-fb4dc06b75fa None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.858940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2feb8-0264-4bac-b068-fb4dc06b75fa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.858940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2feb8-0264-4bac-b068-fb4dc06b75fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.874604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-65f2feb8-0264-4bac-b068-fb4dc06b75fa None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:09.875808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-65f2feb8-0264-4bac-b068-fb4dc06b75fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:09.934815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.956635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 425/856] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:09.958749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2feb8-0264-4bac-b068-fb4dc06b75fa None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.963241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835f66d-8062-4579-ae03-51f8598e9423 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.997758 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.998854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fedf8e-d46e-4e9a-8277-609bb98eef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.000119 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fedf8e-d46e-4e9a-8277-609bb98eef20 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.000654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fedf8e-d46e-4e9a-8277-609bb98eef20 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.001101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fedf8e-d46e-4e9a-8277-609bb98eef20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.012468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fedf8e-d46e-4e9a-8277-609bb98eef20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.013973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fedf8e-d46e-4e9a-8277-609bb98eef20 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b6fedf8e-d46e-4e9a-8277-609bb98eef20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:10.031821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:10.032827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:10.034379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:10.034379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:10.034379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:10.034379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 432/857] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:10.034379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9919a-f34e-4559-b0c1-b28fdb6bcc6e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.034379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9919a-f34e-4559-b0c1-b28fdb6bcc6e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.034731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9919a-f34e-4559-b0c1-b28fdb6bcc6e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.037107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 433/858] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:10.038288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9919a-f34e-4559-b0c1-b28fdb6bcc6e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.089261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2feb8-0264-4bac-b068-fb4dc06b75fa None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.129783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e86006-1c68-4c17-97c8-baf58d266cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.130392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e86006-1c68-4c17-97c8-baf58d266cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.130675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e86006-1c68-4c17-97c8-baf58d266cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.130949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e86006-1c68-4c17-97c8-baf58d266cf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.151390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9919a-f34e-4559-b0c1-b28fdb6bcc6e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.190312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672af09-b2a8-4871-a5a3-af4064016d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.191001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672af09-b2a8-4871-a5a3-af4064016d92 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.191281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672af09-b2a8-4871-a5a3-af4064016d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.191544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672af09-b2a8-4871-a5a3-af4064016d92 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.192209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672af09-b2a8-4871-a5a3-af4064016d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:10.192507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672af09-b2a8-4871-a5a3-af4064016d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.193902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672af09-b2a8-4871-a5a3-af4064016d92 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.195998 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:10.206548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e86006-1c68-4c17-97c8-baf58d266cf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.208311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 434/859] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:10.208424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e86006-1c68-4c17-97c8-baf58d266cf0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.213753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54fa6-30e6-416e-81ce-4edc88d54fd4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.214038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54fa6-30e6-416e-81ce-4edc88d54fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.214307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54fa6-30e6-416e-81ce-4edc88d54fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.232439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f4d54fa6-30e6-416e-81ce-4edc88d54fd4 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:10.233715 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f4d54fa6-30e6-416e-81ce-4edc88d54fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:10.237337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:10.304818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 435/860] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:10.305706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54fa6-30e6-416e-81ce-4edc88d54fd4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.322101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 426/861] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:10.322101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e86006-1c68-4c17-97c8-baf58d266cf0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.416527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54fa6-30e6-416e-81ce-4edc88d54fd4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.422612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e86006-1c68-4c17-97c8-baf58d266cf0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.442392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:10.450920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:10.457452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a450aa-3e2e-4773-8b94-229762a37dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.458072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a450aa-3e2e-4773-8b94-229762a37dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.458353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a450aa-3e2e-4773-8b94-229762a37dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.458618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a450aa-3e2e-4773-8b94-229762a37dd0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.461826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cc5cd-5eb6-4b68-a6a1-596280a07159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.462420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cc5cd-5eb6-4b68-a6a1-596280a07159 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.462703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cc5cd-5eb6-4b68-a6a1-596280a07159 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.462974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cc5cd-5eb6-4b68-a6a1-596280a07159 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.463622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cc5cd-5eb6-4b68-a6a1-596280a07159 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.464968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cc5cd-5eb6-4b68-a6a1-596280a07159 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.465618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a450aa-3e2e-4773-8b94-229762a37dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.467038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a450aa-3e2e-4773-8b94-229762a37dd0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d7a450aa-3e2e-4773-8b94-229762a37dd0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:10.479671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:10.480697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 436/862] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:10.481695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a450aa-3e2e-4773-8b94-229762a37dd0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.497367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 427/863] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:49:10 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:10.497367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cc5cd-5eb6-4b68-a6a1-596280a07159 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.620798 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a450aa-3e2e-4773-8b94-229762a37dd0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.641142 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cc5cd-5eb6-4b68-a6a1-596280a07159 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.663266 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323d51-1d72-4061-a93e-0da7f4aeed72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.664878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323d51-1d72-4061-a93e-0da7f4aeed72 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.664878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323d51-1d72-4061-a93e-0da7f4aeed72 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.665932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323d51-1d72-4061-a93e-0da7f4aeed72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.678540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25323d51-1d72-4061-a93e-0da7f4aeed72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.680623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25323d51-1d72-4061-a93e-0da7f4aeed72 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.681983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1be978-7708-4c4e-bd0d-1d87a62bccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.682590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1be978-7708-4c4e-bd0d-1d87a62bccc1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.682864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1be978-7708-4c4e-bd0d-1d87a62bccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.683130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1be978-7708-4c4e-bd0d-1d87a62bccc1 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.683861 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1be978-7708-4c4e-bd0d-1d87a62bccc1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:10.684163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1be978-7708-4c4e-bd0d-1d87a62bccc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.685593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1be978-7708-4c4e-bd0d-1d87a62bccc1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-25323d51-1d72-4061-a93e-0da7f4aeed72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:10.699461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:10.700661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:10.701736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:10.701736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:10.701736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:10.704120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 437/864] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:10.705048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323d51-1d72-4061-a93e-0da7f4aeed72 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.705755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 428/865] 10.222.0.78 () {56 vars in 1149 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:10.710415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1be978-7708-4c4e-bd0d-1d87a62bccc1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.748166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:10.757764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:10.821858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323d51-1d72-4061-a93e-0da7f4aeed72 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.829026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1be978-7708-4c4e-bd0d-1d87a62bccc1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.860038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a68278-0079-4997-a798-ac37dab414f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.860645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a68278-0079-4997-a798-ac37dab414f7 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.860928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a68278-0079-4997-a798-ac37dab414f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.861231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a68278-0079-4997-a798-ac37dab414f7 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.861882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a68278-0079-4997-a798-ac37dab414f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.863248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a68278-0079-4997-a798-ac37dab414f7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.865888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a2352-f406-4752-85c3-c3d4c8823166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.866486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a2352-f406-4752-85c3-c3d4c8823166 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.866771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a2352-f406-4752-85c3-c3d4c8823166 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.867026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a2352-f406-4752-85c3-c3d4c8823166 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.867657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a2352-f406-4752-85c3-c3d4c8823166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:10.867957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a2352-f406-4752-85c3-c3d4c8823166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.869409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a2352-f406-4752-85c3-c3d4c8823166 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.881935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 438/866] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:49:10 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:10.882366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 429/867] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:10.891480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a2352-f406-4752-85c3-c3d4c8823166 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.891967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a68278-0079-4997-a798-ac37dab414f7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:11.031902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:11.032801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a2352-f406-4752-85c3-c3d4c8823166 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:11.035020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:11.049879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a68278-0079-4997-a798-ac37dab414f7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:11.080457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08d556-cb76-4909-a35b-233144d53b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:11.081508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08d556-cb76-4909-a35b-233144d53b51 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:11.082024 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08d556-cb76-4909-a35b-233144d53b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:11.082480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08d556-cb76-4909-a35b-233144d53b51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:11.090159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08d556-cb76-4909-a35b-233144d53b51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:11.091950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08d556-cb76-4909-a35b-233144d53b51 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:11.092923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa533dfd-6067-4b52-af56-daf4ad792313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:11.094468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa533dfd-6067-4b52-af56-daf4ad792313 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:11.094797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa533dfd-6067-4b52-af56-daf4ad792313 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:11.095049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa533dfd-6067-4b52-af56-daf4ad792313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ba08d556-cb76-4909-a35b-233144d53b51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:11.107507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:11.108993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:11.110269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:11.110269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:11.110269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:11.110269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:11.110269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:11.110269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:11.110269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:11.110269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:11.110269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 430/868] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:11.117244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08d556-cb76-4909-a35b-233144d53b51 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:11.172828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa533dfd-6067-4b52-af56-daf4ad792313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:11.174311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa533dfd-6067-4b52-af56-daf4ad792313 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:11.232082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08d556-cb76-4909-a35b-233144d53b51 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:11.273299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6dfb1-a242-4365-bfd4-b3b9dc3e67b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:11.274382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6dfb1-a242-4365-bfd4-b3b9dc3e67b5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:11.274870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6dfb1-a242-4365-bfd4-b3b9dc3e67b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:11.275339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6dfb1-a242-4365-bfd4-b3b9dc3e67b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:11.276215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6dfb1-a242-4365-bfd4-b3b9dc3e67b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:11.276735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6dfb1-a242-4365-bfd4-b3b9dc3e67b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:11.277923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:11.279662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6dfb1-a242-4365-bfd4-b3b9dc3e67b5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:11.288357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 439/869] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:11.293630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa533dfd-6067-4b52-af56-daf4ad792313 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:11.299891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 431/870] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:11.301439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6dfb1-a242-4365-bfd4-b3b9dc3e67b5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:11.332323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:11.426566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6dfb1-a242-4365-bfd4-b3b9dc3e67b5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:11.437294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa533dfd-6067-4b52-af56-daf4ad792313 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:11.463175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b00701-d9da-4ab0-aad1-63884a10763a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:11.463843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b00701-d9da-4ab0-aad1-63884a10763a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:11.464051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b00701-d9da-4ab0-aad1-63884a10763a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:11.464319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b00701-d9da-4ab0-aad1-63884a10763a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:11.465053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b00701-d9da-4ab0-aad1-63884a10763a None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=c53b9bcd675940c19febbc8fefb1f28b) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:11.465423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b00701-d9da-4ab0-aad1-63884a10763a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:11.466800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b00701-d9da-4ab0-aad1-63884a10763a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:11.477525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 432/871] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:11 2022] GET /identity/v3/role_assignments?user.id=c53b9bcd675940c19febbc8fefb1f28b&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:11.482948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d846dcc-931c-4fbb-9866-02e79b28c7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:11.483659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846dcc-931c-4fbb-9866-02e79b28c7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:11.483910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846dcc-931c-4fbb-9866-02e79b28c7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:11.484182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846dcc-931c-4fbb-9866-02e79b28c7a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:11.496004 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d846dcc-931c-4fbb-9866-02e79b28c7a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:11.497617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d846dcc-931c-4fbb-9866-02e79b28c7a7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-9d846dcc-931c-4fbb-9866-02e79b28c7a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:11.515389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:11.516460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:11.517376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:11.517376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:11.517376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:11.517376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 440/872] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:11.522820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b00701-d9da-4ab0-aad1-63884a10763a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:11.629827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:11.646087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b00701-d9da-4ab0-aad1-63884a10763a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:11.682352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d874d80-9efe-4eab-95b8-55474cbe7721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:11.682954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d874d80-9efe-4eab-95b8-55474cbe7721 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:11.683284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d874d80-9efe-4eab-95b8-55474cbe7721 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:11.683564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d874d80-9efe-4eab-95b8-55474cbe7721 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:11.684203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d874d80-9efe-4eab-95b8-55474cbe7721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:11.684503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d874d80-9efe-4eab-95b8-55474cbe7721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:11.685889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d874d80-9efe-4eab-95b8-55474cbe7721 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:11.699525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 433/873] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:11.705778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d846dcc-931c-4fbb-9866-02e79b28c7a7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:11.755969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:11.814168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d846dcc-931c-4fbb-9866-02e79b28c7a7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:11.851763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026b89b-18e7-40a4-8948-5251fd3b4115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:11.852738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026b89b-18e7-40a4-8948-5251fd3b4115 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:11.853240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026b89b-18e7-40a4-8948-5251fd3b4115 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:11.853705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026b89b-18e7-40a4-8948-5251fd3b4115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:11.860976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026b89b-18e7-40a4-8948-5251fd3b4115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:11.862761 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026b89b-18e7-40a4-8948-5251fd3b4115 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d026b89b-18e7-40a4-8948-5251fd3b4115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:11.874507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:11.875423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:11.876438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:11.876438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:11.876438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:11.876438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:11.876438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:11.876438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:11.876438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:11.876438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:11.878838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 441/874] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:11.884651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d874d80-9efe-4eab-95b8-55474cbe7721 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:11.991124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d874d80-9efe-4eab-95b8-55474cbe7721 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:12.027569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d78444-2276-4764-a27f-b8c20f363c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:12.028543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d78444-2276-4764-a27f-b8c20f363c4c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:12.029015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d78444-2276-4764-a27f-b8c20f363c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:12.029502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d78444-2276-4764-a27f-b8c20f363c4c None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:12.031328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d78444-2276-4764-a27f-b8c20f363c4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:12.031939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d78444-2276-4764-a27f-b8c20f363c4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:12.033590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d78444-2276-4764-a27f-b8c20f363c4c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:12.047691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 434/875] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:12.053968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026b89b-18e7-40a4-8948-5251fd3b4115 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:12.162895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026b89b-18e7-40a4-8948-5251fd3b4115 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:12.165636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:12.200970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7ff46-9566-48b9-b841-706938ee6881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:12.202013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7ff46-9566-48b9-b841-706938ee6881 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:12.203064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7ff46-9566-48b9-b841-706938ee6881 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:12.203536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7ff46-9566-48b9-b841-706938ee6881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:12.204657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7ff46-9566-48b9-b841-706938ee6881 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=308bbd2bc1224533b573c80f0565252a) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:12.205305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7ff46-9566-48b9-b841-706938ee6881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:12.206788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7ff46-9566-48b9-b841-706938ee6881 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:12.217100 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 442/876] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:12 2022] GET /identity/v3/role_assignments?user.id=308bbd2bc1224533b573c80f0565252a&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:12.476082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc01d0d-f40c-4917-bfd6-8894468820b1 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:12.476275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc01d0d-f40c-4917-bfd6-8894468820b1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:12.476539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc01d0d-f40c-4917-bfd6-8894468820b1 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:12.478170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 435/877] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:12.484394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d366f-2866-41ee-9d4c-4acd0d860a40 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:12.484847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d366f-2866-41ee-9d4c-4acd0d860a40 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:12.485167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d366f-2866-41ee-9d4c-4acd0d860a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:12.503403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c11d366f-2866-41ee-9d4c-4acd0d860a40 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:12.504596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c11d366f-2866-41ee-9d4c-4acd0d860a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:12.581208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 443/878] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:12.847667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc01d0d-f40c-4917-bfd6-8894468820b1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:12.857557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:12.936303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e882d-7ebb-4e2d-af5e-950a01cecddd None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:12.937265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e882d-7ebb-4e2d-af5e-950a01cecddd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:12.937489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e882d-7ebb-4e2d-af5e-950a01cecddd None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:12.939096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 444/879] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:12.944220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00c714-46bd-485b-9448-b48ad786e296 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:12.944759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00c714-46bd-485b-9448-b48ad786e296 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:12.945263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00c714-46bd-485b-9448-b48ad786e296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:12.963113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc01d0d-f40c-4917-bfd6-8894468820b1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:12.968830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7a00c714-46bd-485b-9448-b48ad786e296 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:12.970360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7a00c714-46bd-485b-9448-b48ad786e296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:13.001327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916003f0-4363-413a-a7da-2752c24334a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:13.002329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916003f0-4363-413a-a7da-2752c24334a6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.002812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916003f0-4363-413a-a7da-2752c24334a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.003281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916003f0-4363-413a-a7da-2752c24334a6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.004077 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916003f0-4363-413a-a7da-2752c24334a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:13.005729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916003f0-4363-413a-a7da-2752c24334a6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-916003f0-4363-413a-a7da-2752c24334a6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:13.015726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:13.016640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:13.017574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:13.024906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 436/880] 10.222.0.78 () {56 vars in 1132 bytes} [Tue Aug 30 00:49:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:13.026371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916003f0-4363-413a-a7da-2752c24334a6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:13.057164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 445/881] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:13.108723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:13.143209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916003f0-4363-413a-a7da-2752c24334a6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:13.184385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cb3fb-a66e-4c49-b9ee-fc9b8e758387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:13.184989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cb3fb-a66e-4c49-b9ee-fc9b8e758387 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.185307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cb3fb-a66e-4c49-b9ee-fc9b8e758387 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.185582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cb3fb-a66e-4c49-b9ee-fc9b8e758387 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.186212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cb3fb-a66e-4c49-b9ee-fc9b8e758387 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:13.186545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cb3fb-a66e-4c49-b9ee-fc9b8e758387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:13.188101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cb3fb-a66e-4c49-b9ee-fc9b8e758387 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:13.193291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 437/882] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:13.199206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00c714-46bd-485b-9448-b48ad786e296 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:13.228799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:13.319641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00c714-46bd-485b-9448-b48ad786e296 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:13.356765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee986af-9168-4ab5-9af7-7c7c7ccff98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:13.357406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee986af-9168-4ab5-9af7-7c7c7ccff98b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.357687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee986af-9168-4ab5-9af7-7c7c7ccff98b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.357961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee986af-9168-4ab5-9af7-7c7c7ccff98b None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.358679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee986af-9168-4ab5-9af7-7c7c7ccff98b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:13.359194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee986af-9168-4ab5-9af7-7c7c7ccff98b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:13.360627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee986af-9168-4ab5-9af7-7c7c7ccff98b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:13.366094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 446/883] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:13.367308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cb3fb-a66e-4c49-b9ee-fc9b8e758387 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:13.372272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee986af-9168-4ab5-9af7-7c7c7ccff98b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:13.481444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:13.487590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cb3fb-a66e-4c49-b9ee-fc9b8e758387 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:13.491804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee986af-9168-4ab5-9af7-7c7c7ccff98b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:13.526292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61129412-680c-41d4-8831-3de9d7840dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:13.527294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61129412-680c-41d4-8831-3de9d7840dab None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.527871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61129412-680c-41d4-8831-3de9d7840dab None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.528426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61129412-680c-41d4-8831-3de9d7840dab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.529217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61129412-680c-41d4-8831-3de9d7840dab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:13.530328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3c1fd-2dd6-44d8-8bd0-91def6a4e5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:13.531460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3c1fd-2dd6-44d8-8bd0-91def6a4e5bf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.531777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3c1fd-2dd6-44d8-8bd0-91def6a4e5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.532061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61129412-680c-41d4-8831-3de9d7840dab None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:13.532209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3c1fd-2dd6-44d8-8bd0-91def6a4e5bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.532909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3c1fd-2dd6-44d8-8bd0-91def6a4e5bf None admin] RBAC: Adding query filter params (interface=public, service_id=8e59ddf7c9e3479bb19bfff6ce4b91c9, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:13.533262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3c1fd-2dd6-44d8-8bd0-91def6a4e5bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:13.534635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:13.535664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3c1fd-2dd6-44d8-8bd0-91def6a4e5bf None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:13.541449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 447/884] 10.222.0.78 () {56 vars in 1251 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/endpoints?service_id=8e59ddf7c9e3479bb19bfff6ce4b91c9&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-61129412-680c-41d4-8831-3de9d7840dab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:13.543669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:13.544652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:13.545517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:13.546996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 438/885] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:13.551904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3c1fd-2dd6-44d8-8bd0-91def6a4e5bf None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:13.552689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61129412-680c-41d4-8831-3de9d7840dab None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:13.653023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3c1fd-2dd6-44d8-8bd0-91def6a4e5bf None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:13.665808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61129412-680c-41d4-8831-3de9d7840dab None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:13.690014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080e8d-9780-459e-9b8d-9cd4179f0493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:13.690866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080e8d-9780-459e-9b8d-9cd4179f0493 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.691340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080e8d-9780-459e-9b8d-9cd4179f0493 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.691783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080e8d-9780-459e-9b8d-9cd4179f0493 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.692569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080e8d-9780-459e-9b8d-9cd4179f0493 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:13.694389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080e8d-9780-459e-9b8d-9cd4179f0493 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:13.703546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69751a-520d-475a-8a41-d9d7d3d3fc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:13.704152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69751a-520d-475a-8a41-d9d7d3d3fc63 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.704430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69751a-520d-475a-8a41-d9d7d3d3fc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.704698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69751a-520d-475a-8a41-d9d7d3d3fc63 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.705429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69751a-520d-475a-8a41-d9d7d3d3fc63 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:13.705687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 448/886] 10.222.0.78 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:49:13.705947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69751a-520d-475a-8a41-d9d7d3d3fc63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:13.707492 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69751a-520d-475a-8a41-d9d7d3d3fc63 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:13.707855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0e802-230e-4e75-b38f-9d9f8b21a551 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.708366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0e802-230e-4e75-b38f-9d9f8b21a551 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.709220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0e802-230e-4e75-b38f-9d9f8b21a551 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.711960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 449/887] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:13.714881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 439/888] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:13.717214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55604dfd-f3bb-45aa-87c1-11f1e612c320 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.717502 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55604dfd-f3bb-45aa-87c1-11f1e612c320 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.718390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55604dfd-f3bb-45aa-87c1-11f1e612c320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.720897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69751a-520d-475a-8a41-d9d7d3d3fc63 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:13.743116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-55604dfd-f3bb-45aa-87c1-11f1e612c320 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:13.744796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-55604dfd-f3bb-45aa-87c1-11f1e612c320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:13.767698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:13.801757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:13.845567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 450/889] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:13.855974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69751a-520d-475a-8a41-d9d7d3d3fc63 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:13.894580 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afbfc8-34c0-4609-930a-df41c609caa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:13.895189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afbfc8-34c0-4609-930a-df41c609caa7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.895462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afbfc8-34c0-4609-930a-df41c609caa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.895728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afbfc8-34c0-4609-930a-df41c609caa7 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.896355 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afbfc8-34c0-4609-930a-df41c609caa7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:13.896655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afbfc8-34c0-4609-930a-df41c609caa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:13.898021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afbfc8-34c0-4609-930a-df41c609caa7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:13.903409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 440/890] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:13.909790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55604dfd-f3bb-45aa-87c1-11f1e612c320 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.022320 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55604dfd-f3bb-45aa-87c1-11f1e612c320 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.057918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbab6ee-3fbc-4e99-8e4c-2321ae601700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.058897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbab6ee-3fbc-4e99-8e4c-2321ae601700 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.059370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbab6ee-3fbc-4e99-8e4c-2321ae601700 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.059831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbab6ee-3fbc-4e99-8e4c-2321ae601700 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.060775 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbab6ee-3fbc-4e99-8e4c-2321ae601700 None admin] RBAC: Adding query filter params (interface=public, service_id=fcc28d604b4646e1839758d47827cef7, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:14.061350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbab6ee-3fbc-4e99-8e4c-2321ae601700 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.063562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbab6ee-3fbc-4e99-8e4c-2321ae601700 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.069984 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 451/891] 10.222.0.78 () {56 vars in 1251 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/endpoints?service_id=fcc28d604b4646e1839758d47827cef7&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:14.077228 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afbfc8-34c0-4609-930a-df41c609caa7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.129046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbab6ee-3fbc-4e99-8e4c-2321ae601700 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.145820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.193308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afbfc8-34c0-4609-930a-df41c609caa7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.227291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.239308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5776b-4929-4e50-9fe4-d34f9dc8679f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.240062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5776b-4929-4e50-9fe4-d34f9dc8679f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.240349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5776b-4929-4e50-9fe4-d34f9dc8679f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.240625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5776b-4929-4e50-9fe4-d34f9dc8679f None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.241596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5776b-4929-4e50-9fe4-d34f9dc8679f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.243266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5776b-4929-4e50-9fe4-d34f9dc8679f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.251537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 441/892] 10.222.0.78 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:49:14.257324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbab6ee-3fbc-4e99-8e4c-2321ae601700 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.296812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998609d-0cc9-4ee8-9cbb-fa98ce0a8719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.297846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998609d-0cc9-4ee8-9cbb-fa98ce0a8719 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.298273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998609d-0cc9-4ee8-9cbb-fa98ce0a8719 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.298685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998609d-0cc9-4ee8-9cbb-fa98ce0a8719 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.299437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998609d-0cc9-4ee8-9cbb-fa98ce0a8719 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.301034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998609d-0cc9-4ee8-9cbb-fa98ce0a8719 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-2998609d-0cc9-4ee8-9cbb-fa98ce0a8719 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:14.311141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:14.313288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:14.314185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:14.316655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 452/893] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:14.323042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5776b-4929-4e50-9fe4-d34f9dc8679f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.342493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d0c68-aa40-4f27-94d0-83242cdf824e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.342885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d0c68-aa40-4f27-94d0-83242cdf824e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.343382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d0c68-aa40-4f27-94d0-83242cdf824e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.345914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 453/894] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:14.350355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f48c0-12d4-4238-ae73-5792de7be582 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.350640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f48c0-12d4-4238-ae73-5792de7be582 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.350912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f48c0-12d4-4238-ae73-5792de7be582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.370337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a88f48c0-12d4-4238-ae73-5792de7be582 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:14.371557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a88f48c0-12d4-4238-ae73-5792de7be582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:14.458902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5776b-4929-4e50-9fe4-d34f9dc8679f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.461185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 454/895] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:14.499217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d197e-e733-4619-8471-14badf4ba225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.499881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d197e-e733-4619-8471-14badf4ba225 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.500192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d197e-e733-4619-8471-14badf4ba225 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.500465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d197e-e733-4619-8471-14badf4ba225 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.501104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d197e-e733-4619-8471-14badf4ba225 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:14.501423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d197e-e733-4619-8471-14badf4ba225 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.502819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d197e-e733-4619-8471-14badf4ba225 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.508084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 442/896] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:14.513635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f48c0-12d4-4238-ae73-5792de7be582 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.526352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.626646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f48c0-12d4-4238-ae73-5792de7be582 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.662171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b4ace-8ad1-4f23-b1ea-5e5aa4e65310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.662779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b4ace-8ad1-4f23-b1ea-5e5aa4e65310 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.663057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b4ace-8ad1-4f23-b1ea-5e5aa4e65310 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.663329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b4ace-8ad1-4f23-b1ea-5e5aa4e65310 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.663954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b4ace-8ad1-4f23-b1ea-5e5aa4e65310 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:14.664253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b4ace-8ad1-4f23-b1ea-5e5aa4e65310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.665659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b4ace-8ad1-4f23-b1ea-5e5aa4e65310 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.670550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 455/897] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:14.732549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d197e-e733-4619-8471-14badf4ba225 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.801691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.851390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d197e-e733-4619-8471-14badf4ba225 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.889274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b5b63-5ddd-4c63-9b1f-6fea9be52536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.890470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5b63-5ddd-4c63-9b1f-6fea9be52536 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.890979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5b63-5ddd-4c63-9b1f-6fea9be52536 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.891371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5b63-5ddd-4c63-9b1f-6fea9be52536 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.902352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b5b63-5ddd-4c63-9b1f-6fea9be52536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9d3b5b63-5ddd-4c63-9b1f-6fea9be52536 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:14.908196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:14.909133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:14.909965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 443/898] 10.222.0.78 () {56 vars in 1120 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:14.915495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b4ace-8ad1-4f23-b1ea-5e5aa4e65310 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.995480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:15.037108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b4ace-8ad1-4f23-b1ea-5e5aa4e65310 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.071613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f131ff7-287e-4e92-bae3-9e9b0c6b9308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.072226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f131ff7-287e-4e92-bae3-9e9b0c6b9308 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.072507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f131ff7-287e-4e92-bae3-9e9b0c6b9308 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.072781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f131ff7-287e-4e92-bae3-9e9b0c6b9308 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.073484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f131ff7-287e-4e92-bae3-9e9b0c6b9308 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:15.073787 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f131ff7-287e-4e92-bae3-9e9b0c6b9308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:15.075157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f131ff7-287e-4e92-bae3-9e9b0c6b9308 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:15.084406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 456/899] 10.222.0.78 () {56 vars in 1115 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:15.089793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b5b63-5ddd-4c63-9b1f-6fea9be52536 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:15.195367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b5b63-5ddd-4c63-9b1f-6fea9be52536 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.209954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:15.230579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efc9d1-ee37-477b-94c3-10b4faaff9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.231200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efc9d1-ee37-477b-94c3-10b4faaff9ab None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.231474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efc9d1-ee37-477b-94c3-10b4faaff9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.231741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efc9d1-ee37-477b-94c3-10b4faaff9ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.242951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efc9d1-ee37-477b-94c3-10b4faaff9ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:15.244388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efc9d1-ee37-477b-94c3-10b4faaff9ab None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-e0efc9d1-ee37-477b-94c3-10b4faaff9ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:15.259962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:15.260874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:15.261784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:15.261784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:15.261784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:15.261784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 444/900] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:15.268106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f131ff7-287e-4e92-bae3-9e9b0c6b9308 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:15.383732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f131ff7-287e-4e92-bae3-9e9b0c6b9308 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.419744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412b228-15be-4ea0-bbaf-e30557d13689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.420359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412b228-15be-4ea0-bbaf-e30557d13689 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.420640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412b228-15be-4ea0-bbaf-e30557d13689 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.421147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412b228-15be-4ea0-bbaf-e30557d13689 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.421926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412b228-15be-4ea0-bbaf-e30557d13689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:15.422287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412b228-15be-4ea0-bbaf-e30557d13689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:15.423663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412b228-15be-4ea0-bbaf-e30557d13689 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:15.425194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:15.437038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 457/901] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:15.442889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efc9d1-ee37-477b-94c3-10b4faaff9ab None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:15.545136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efc9d1-ee37-477b-94c3-10b4faaff9ab None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.582745 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39241a2-016b-4e95-869a-2aabe8bbed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.583382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39241a2-016b-4e95-869a-2aabe8bbed6e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.583661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39241a2-016b-4e95-869a-2aabe8bbed6e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.583929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39241a2-016b-4e95-869a-2aabe8bbed6e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.591655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39241a2-016b-4e95-869a-2aabe8bbed6e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:15.593200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39241a2-016b-4e95-869a-2aabe8bbed6e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-b39241a2-016b-4e95-869a-2aabe8bbed6e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:15.604152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:15.606227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:15.609056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:15.610347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 445/902] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:49:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:15.617095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412b228-15be-4ea0-bbaf-e30557d13689 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:15.723900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412b228-15be-4ea0-bbaf-e30557d13689 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.759283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40c062-02cd-41b6-b426-6c4a54edb2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.759895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40c062-02cd-41b6-b426-6c4a54edb2fd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.760175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40c062-02cd-41b6-b426-6c4a54edb2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.760449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40c062-02cd-41b6-b426-6c4a54edb2fd None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.761214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40c062-02cd-41b6-b426-6c4a54edb2fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:15.761523 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40c062-02cd-41b6-b426-6c4a54edb2fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:15.762961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40c062-02cd-41b6-b426-6c4a54edb2fd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:15.782920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 458/903] 10.222.0.78 () {56 vars in 1149 bytes} [Tue Aug 30 00:49:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:15.788380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39241a2-016b-4e95-869a-2aabe8bbed6e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:15.813317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a3ae0-4eb4-42df-9e59-af110d9fb04d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.813621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a3ae0-4eb4-42df-9e59-af110d9fb04d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.813788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a3ae0-4eb4-42df-9e59-af110d9fb04d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.815359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 459/904] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:15.820253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:15.820455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f09dc0-daa1-47de-bc79-91d004abb43d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.820672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f09dc0-daa1-47de-bc79-91d004abb43d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.820946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f09dc0-daa1-47de-bc79-91d004abb43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.839950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-02f09dc0-daa1-47de-bc79-91d004abb43d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:15.841154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-02f09dc0-daa1-47de-bc79-91d004abb43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:15.894248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:15.906235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39241a2-016b-4e95-869a-2aabe8bbed6e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.923130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 460/905] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:15.950070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c7fa4-3c21-4826-aaa0-c3815aec7aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.950729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c7fa4-3c21-4826-aaa0-c3815aec7aec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.951010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c7fa4-3c21-4826-aaa0-c3815aec7aec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.951279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c7fa4-3c21-4826-aaa0-c3815aec7aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.025347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c7fa4-3c21-4826-aaa0-c3815aec7aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:16.026789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c7fa4-3c21-4826-aaa0-c3815aec7aec None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:16.054245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:16.130929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 446/906] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:15 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:16.136545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f09dc0-daa1-47de-bc79-91d004abb43d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:16.220623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c7fa4-3c21-4826-aaa0-c3815aec7aec None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:16.260825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f09dc0-daa1-47de-bc79-91d004abb43d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:16.298522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d451b-e97e-45f2-9460-61c5a26702fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:16.299535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d451b-e97e-45f2-9460-61c5a26702fa None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.300014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d451b-e97e-45f2-9460-61c5a26702fa None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.300505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d451b-e97e-45f2-9460-61c5a26702fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.312828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d451b-e97e-45f2-9460-61c5a26702fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:16.314378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d451b-e97e-45f2-9460-61c5a26702fa None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:16.329747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c7fa4-3c21-4826-aaa0-c3815aec7aec None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-369d451b-e97e-45f2-9460-61c5a26702fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:16.330934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:16.331828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:16.332700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:16.332700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:16.332700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:16.332700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 461/907] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:16.333264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e22dd-1748-49a7-ba0c-e8fde8f2f89d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.333710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e22dd-1748-49a7-ba0c-e8fde8f2f89d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.334131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e22dd-1748-49a7-ba0c-e8fde8f2f89d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.335810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 462/908] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:16.337561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e22dd-1748-49a7-ba0c-e8fde8f2f89d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:16.365982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:16.376481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b78a65-20e4-41f4-9ba3-2a79e65021ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:16.377131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78a65-20e4-41f4-9ba3-2a79e65021ed None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.377415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78a65-20e4-41f4-9ba3-2a79e65021ed None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.377691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78a65-20e4-41f4-9ba3-2a79e65021ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.378056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:16.378270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78a65-20e4-41f4-9ba3-2a79e65021ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:16.379624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78a65-20e4-41f4-9ba3-2a79e65021ed None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-54b78a65-20e4-41f4-9ba3-2a79e65021ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:16.389165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:16.390249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:16.391157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 447/909] 10.222.0.78 () {56 vars in 1132 bytes} [Tue Aug 30 00:49:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:16.391969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db56c8a-7ce6-433e-b653-9bedb65c71b7 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.391969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db56c8a-7ce6-433e-b653-9bedb65c71b7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.391969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db56c8a-7ce6-433e-b653-9bedb65c71b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.410637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9db56c8a-7ce6-433e-b653-9bedb65c71b7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:16.411826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9db56c8a-7ce6-433e-b653-9bedb65c71b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:16.455194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e22dd-1748-49a7-ba0c-e8fde8f2f89d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:16.485823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 448/910] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:16.490172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db56c8a-7ce6-433e-b653-9bedb65c71b7 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:16.491043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39063c5c-f43f-4b02-baf6-506d0c069e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:16.491653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063c5c-f43f-4b02-baf6-506d0c069e08 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.491932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063c5c-f43f-4b02-baf6-506d0c069e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.492205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063c5c-f43f-4b02-baf6-506d0c069e08 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.492849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39063c5c-f43f-4b02-baf6-506d0c069e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:16.493180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39063c5c-f43f-4b02-baf6-506d0c069e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:16.494551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39063c5c-f43f-4b02-baf6-506d0c069e08 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:16.511518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 463/911] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:16.512591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39063c5c-f43f-4b02-baf6-506d0c069e08 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:16.606672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db56c8a-7ce6-433e-b653-9bedb65c71b7 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:16.629413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:16.642332 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:16.645868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba4023-a915-479e-916b-899a5317fe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:16.646526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba4023-a915-479e-916b-899a5317fe0d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.649206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba4023-a915-479e-916b-899a5317fe0d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.649206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba4023-a915-479e-916b-899a5317fe0d None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.649206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba4023-a915-479e-916b-899a5317fe0d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:16.649206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba4023-a915-479e-916b-899a5317fe0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:16.649524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba4023-a915-479e-916b-899a5317fe0d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:16.651790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39063c5c-f43f-4b02-baf6-506d0c069e08 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:16.660944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 449/912] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:16.662025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba4023-a915-479e-916b-899a5317fe0d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:16.701359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467568a-28a8-433e-a681-881676faea66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:16.703433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467568a-28a8-433e-a681-881676faea66 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.703942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467568a-28a8-433e-a681-881676faea66 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.704404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467568a-28a8-433e-a681-881676faea66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.713786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467568a-28a8-433e-a681-881676faea66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:16.716231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467568a-28a8-433e-a681-881676faea66 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-2467568a-28a8-433e-a681-881676faea66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:16.733060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:16.734654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:16.735552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:16.735552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:16.735552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:16.735552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:16.735552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:16.735552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:16.735552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:16.735552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:16.739714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 464/913] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:16.740832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0262d4-30cd-43e6-8bce-fedaee797b11 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.741160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0262d4-30cd-43e6-8bce-fedaee797b11 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.741438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0262d4-30cd-43e6-8bce-fedaee797b11 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.742852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 465/914] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:16.743538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0262d4-30cd-43e6-8bce-fedaee797b11 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:16.772052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba4023-a915-479e-916b-899a5317fe0d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:16.807174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b88c1e-a5b1-4277-a173-ef30d74bbdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:16.807773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b88c1e-a5b1-4277-a173-ef30d74bbdef None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.808053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b88c1e-a5b1-4277-a173-ef30d74bbdef None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.808326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b88c1e-a5b1-4277-a173-ef30d74bbdef None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.808953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b88c1e-a5b1-4277-a173-ef30d74bbdef None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:16.809286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b88c1e-a5b1-4277-a173-ef30d74bbdef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:16.810638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b88c1e-a5b1-4277-a173-ef30d74bbdef None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:16.815488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 450/915] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:16.816641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853a5db-987b-495b-90da-425dfc1501f6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.816928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853a5db-987b-495b-90da-425dfc1501f6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.817234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853a5db-987b-495b-90da-425dfc1501f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.835133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9853a5db-987b-495b-90da-425dfc1501f6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:16.836321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9853a5db-987b-495b-90da-425dfc1501f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:16.845362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0262d4-30cd-43e6-8bce-fedaee797b11 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:16.881045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:16.881240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d4a83-d60f-48c7-9733-30910e5d24ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:16.881768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d4a83-d60f-48c7-9733-30910e5d24ca None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.882089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d4a83-d60f-48c7-9733-30910e5d24ca None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.882405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d4a83-d60f-48c7-9733-30910e5d24ca None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.883387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d4a83-d60f-48c7-9733-30910e5d24ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:16.883687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d4a83-d60f-48c7-9733-30910e5d24ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:16.885217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d4a83-d60f-48c7-9733-30910e5d24ca None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:16.896969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 466/916] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:16.897549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d4a83-d60f-48c7-9733-30910e5d24ca None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:16.909739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 451/917] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:16.910554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853a5db-987b-495b-90da-425dfc1501f6 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:16.913681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:17.001592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d4a83-d60f-48c7-9733-30910e5d24ca None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:17.008020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853a5db-987b-495b-90da-425dfc1501f6 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:17.039853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72fb45-95aa-45de-b381-0d47cddeef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:17.040483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72fb45-95aa-45de-b381-0d47cddeef9a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.040759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72fb45-95aa-45de-b381-0d47cddeef9a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.041025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72fb45-95aa-45de-b381-0d47cddeef9a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.041636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72fb45-95aa-45de-b381-0d47cddeef9a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:17.042996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72fb45-95aa-45de-b381-0d47cddeef9a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:17.044887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b2ab-724e-47bc-b85b-9f90d0f346ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:17.046671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b2ab-724e-47bc-b85b-9f90d0f346ce None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.047154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b2ab-724e-47bc-b85b-9f90d0f346ce None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.047615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b2ab-724e-47bc-b85b-9f90d0f346ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.048467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1b2ab-724e-47bc-b85b-9f90d0f346ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:17.050139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1b2ab-724e-47bc-b85b-9f90d0f346ce None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ad72fb45-95aa-45de-b381-0d47cddeef9a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:17.052401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:17.053470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:17.054294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:17.058518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 467/918] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:17.059204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72fb45-95aa-45de-b381-0d47cddeef9a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:17.071646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 452/919] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:16 2022] POST /identity/v3/endpoints => generated 347 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:17.071646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b2ab-724e-47bc-b85b-9f90d0f346ce None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:17.127378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:17.172499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:17.198017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72fb45-95aa-45de-b381-0d47cddeef9a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:17.218825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b2ab-724e-47bc-b85b-9f90d0f346ce None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:17.243948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3b6fd-5b62-4049-8add-b03054423638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:17.244610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3b6fd-5b62-4049-8add-b03054423638 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.244893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3b6fd-5b62-4049-8add-b03054423638 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.245200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3b6fd-5b62-4049-8add-b03054423638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.245931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3b6fd-5b62-4049-8add-b03054423638 None admin] RBAC: Adding query filter params (role.id=c0568265539e4d40868747758cd54de3, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=308bbd2bc1224533b573c80f0565252a) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:17.247151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3b6fd-5b62-4049-8add-b03054423638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:17.247669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3b6fd-5b62-4049-8add-b03054423638 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:17.254351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 468/920] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:17 2022] GET /identity/v3/role_assignments?user.id=308bbd2bc1224533b573c80f0565252a&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=c0568265539e4d40868747758cd54de3 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:17.254968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3b6fd-5b62-4049-8add-b03054423638 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:17.267337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46650b-e0e8-4a41-b8ef-320c18426d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:17.267976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46650b-e0e8-4a41-b8ef-320c18426d2b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.268257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46650b-e0e8-4a41-b8ef-320c18426d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.268528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46650b-e0e8-4a41-b8ef-320c18426d2b None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.269236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46650b-e0e8-4a41-b8ef-320c18426d2b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:17.269536 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46650b-e0e8-4a41-b8ef-320c18426d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:17.271645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46650b-e0e8-4a41-b8ef-320c18426d2b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:17.278891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 453/921] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:17.285918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46650b-e0e8-4a41-b8ef-320c18426d2b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:17.394092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3b6fd-5b62-4049-8add-b03054423638 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:17.433821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f4b5e-995f-460c-accb-1f9a9effbaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:17.434358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f4b5e-995f-460c-accb-1f9a9effbaea None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.434637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f4b5e-995f-460c-accb-1f9a9effbaea None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.434903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f4b5e-995f-460c-accb-1f9a9effbaea None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.435555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f4b5e-995f-460c-accb-1f9a9effbaea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:17.438688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f4b5e-995f-460c-accb-1f9a9effbaea None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:17.440415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:17.448911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46650b-e0e8-4a41-b8ef-320c18426d2b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:17.466017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 469/922] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:49:17 2022] POST /identity/v3/services => generated 241 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:17.505884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e9b9c-c1a5-4ca7-8542-f9d86ffe84a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:17.506785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e9b9c-c1a5-4ca7-8542-f9d86ffe84a8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.507222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e9b9c-c1a5-4ca7-8542-f9d86ffe84a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.507646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e9b9c-c1a5-4ca7-8542-f9d86ffe84a8 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.508484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e9b9c-c1a5-4ca7-8542-f9d86ffe84a8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:17.508940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e9b9c-c1a5-4ca7-8542-f9d86ffe84a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:17.510703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e9b9c-c1a5-4ca7-8542-f9d86ffe84a8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:17.521471 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 454/923] 10.222.0.78 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:17.524062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f4b5e-995f-460c-accb-1f9a9effbaea None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:17.527849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:17.641953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f4b5e-995f-460c-accb-1f9a9effbaea None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:17.677705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72f157-8709-48fa-a62b-355912a9369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:17.679043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72f157-8709-48fa-a62b-355912a9369e None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.679486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72f157-8709-48fa-a62b-355912a9369e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.679908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72f157-8709-48fa-a62b-355912a9369e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.680751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72f157-8709-48fa-a62b-355912a9369e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:17.682298 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72f157-8709-48fa-a62b-355912a9369e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:17.713286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 470/924] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:19.087972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5b80b-ba57-4427-bb90-258b415a074b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.088538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5b80b-ba57-4427-bb90-258b415a074b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.088538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5b80b-ba57-4427-bb90-258b415a074b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.090113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 455/925] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:19.096181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3451b-2744-4e9d-b534-982fdfbcf44e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.096620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3451b-2744-4e9d-b534-982fdfbcf44e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.096707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3451b-2744-4e9d-b534-982fdfbcf44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.115849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c6b3451b-2744-4e9d-b534-982fdfbcf44e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:19.117030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c6b3451b-2744-4e9d-b534-982fdfbcf44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:19.172154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.193142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 471/926] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:19.260813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d8421-6873-4f30-a27b-61443a782cbb None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.261344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d8421-6873-4f30-a27b-61443a782cbb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.261832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d8421-6873-4f30-a27b-61443a782cbb None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.263601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 456/927] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:19.270844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd1c02-564a-45ed-9f0f-62f7969c8ff1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.271376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd1c02-564a-45ed-9f0f-62f7969c8ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.271877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd1c02-564a-45ed-9f0f-62f7969c8ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.290398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4fbd1c02-564a-45ed-9f0f-62f7969c8ff1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:19.291906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4fbd1c02-564a-45ed-9f0f-62f7969c8ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:19.361514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 472/928] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:19.476695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c27523-fd96-40da-9437-f27ddbc0a74f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.477380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c27523-fd96-40da-9437-f27ddbc0a74f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.477865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c27523-fd96-40da-9437-f27ddbc0a74f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.479455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd1c02-564a-45ed-9f0f-62f7969c8ff1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.479681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 457/929] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:19.485312 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99974b5-4b6e-4fc0-97f8-4f423277c540 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.485917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99974b5-4b6e-4fc0-97f8-4f423277c540 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.486391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99974b5-4b6e-4fc0-97f8-4f423277c540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.510501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f99974b5-4b6e-4fc0-97f8-4f423277c540 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:19.513166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f99974b5-4b6e-4fc0-97f8-4f423277c540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:19.594963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd1c02-564a-45ed-9f0f-62f7969c8ff1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.605450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 458/930] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:19.610978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.632550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94f8e8-21a5-4182-8070-2c90d31cb960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.633207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94f8e8-21a5-4182-8070-2c90d31cb960 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.633483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94f8e8-21a5-4182-8070-2c90d31cb960 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.633748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94f8e8-21a5-4182-8070-2c90d31cb960 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.634323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94f8e8-21a5-4182-8070-2c90d31cb960 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.635690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94f8e8-21a5-4182-8070-2c90d31cb960 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.636339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99974b5-4b6e-4fc0-97f8-4f423277c540 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ff94f8e8-21a5-4182-8070-2c90d31cb960 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:19.651861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:19.652949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:19.654181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 473/931] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:19.662033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94f8e8-21a5-4182-8070-2c90d31cb960 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.746437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.764194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99974b5-4b6e-4fc0-97f8-4f423277c540 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.787504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94f8e8-21a5-4182-8070-2c90d31cb960 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.800440 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4c1bd-4e18-4471-abc8-a57d773102f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.801048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4c1bd-4e18-4471-abc8-a57d773102f1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.801379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4c1bd-4e18-4471-abc8-a57d773102f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.801619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4c1bd-4e18-4471-abc8-a57d773102f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.813547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4c1bd-4e18-4471-abc8-a57d773102f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-70e4c1bd-4e18-4471-abc8-a57d773102f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:19.819434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:19.820350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:19.821245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:19.822236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 459/932] 10.222.0.78 () {56 vars in 1120 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:19.824435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0891fd2-30b8-4221-a174-e608694229d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.825041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0891fd2-30b8-4221-a174-e608694229d8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.825346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0891fd2-30b8-4221-a174-e608694229d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.825769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0891fd2-30b8-4221-a174-e608694229d8 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.826397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0891fd2-30b8-4221-a174-e608694229d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:19.826754 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0891fd2-30b8-4221-a174-e608694229d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.829013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43f9e-f652-4973-b215-1fb846965982 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.829160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0891fd2-30b8-4221-a174-e608694229d8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.829320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43f9e-f652-4973-b215-1fb846965982 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.829591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43f9e-f652-4973-b215-1fb846965982 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.833123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 460/933] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:19.833123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43f9e-f652-4973-b215-1fb846965982 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.837300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 474/934] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:19.837300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0d85d-7198-4ea3-8993-b4ca45257f96 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.837300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0d85d-7198-4ea3-8993-b4ca45257f96 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.837796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0d85d-7198-4ea3-8993-b4ca45257f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.857400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ade0d85d-7198-4ea3-8993-b4ca45257f96 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:19.859186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ade0d85d-7198-4ea3-8993-b4ca45257f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:19.897563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.929064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 475/935] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:19.929920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0d85d-7198-4ea3-8993-b4ca45257f96 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.944996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43f9e-f652-4973-b215-1fb846965982 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.980006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3359-81d0-429a-84a8-5c256caae877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.980604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3359-81d0-429a-84a8-5c256caae877 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.980887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3359-81d0-429a-84a8-5c256caae877 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.981186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3359-81d0-429a-84a8-5c256caae877 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.981850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc3359-81d0-429a-84a8-5c256caae877 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:19.982153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc3359-81d0-429a-84a8-5c256caae877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.983545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc3359-81d0-429a-84a8-5c256caae877 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.991796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 461/936] 10.222.0.78 () {56 vars in 1115 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:19.992576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3359-81d0-429a-84a8-5c256caae877 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.003664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.042554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0d85d-7198-4ea3-8993-b4ca45257f96 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.087679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52ae33-79a6-4915-ae3f-f439dee2c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.088681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52ae33-79a6-4915-ae3f-f439dee2c44e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.089546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52ae33-79a6-4915-ae3f-f439dee2c44e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.090232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52ae33-79a6-4915-ae3f-f439dee2c44e None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.091006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52ae33-79a6-4915-ae3f-f439dee2c44e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.091635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52ae33-79a6-4915-ae3f-f439dee2c44e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.093934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52ae33-79a6-4915-ae3f-f439dee2c44e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.098487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3359-81d0-429a-84a8-5c256caae877 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.098956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 476/937] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:20.100445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52ae33-79a6-4915-ae3f-f439dee2c44e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.147168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355cab6-21f9-43d3-969e-50f1ec20e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.147862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355cab6-21f9-43d3-969e-50f1ec20e41c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.148078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355cab6-21f9-43d3-969e-50f1ec20e41c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.148406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355cab6-21f9-43d3-969e-50f1ec20e41c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.148997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355cab6-21f9-43d3-969e-50f1ec20e41c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.150576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355cab6-21f9-43d3-969e-50f1ec20e41c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.151582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-4355cab6-21f9-43d3-969e-50f1ec20e41c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:20.162421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:20.163509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:20.164983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 462/938] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:20.169574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355cab6-21f9-43d3-969e-50f1ec20e41c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.243423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52ae33-79a6-4915-ae3f-f439dee2c44e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.261279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.287579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc840c42-ed9a-471f-8500-106e2daba416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.288475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc840c42-ed9a-471f-8500-106e2daba416 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.288752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc840c42-ed9a-471f-8500-106e2daba416 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.289020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc840c42-ed9a-471f-8500-106e2daba416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.292881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355cab6-21f9-43d3-969e-50f1ec20e41c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.303283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc840c42-ed9a-471f-8500-106e2daba416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.305241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc840c42-ed9a-471f-8500-106e2daba416 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-bc840c42-ed9a-471f-8500-106e2daba416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:20.323610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:20.324578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:20.330588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:20.330588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:20.330588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:20.330588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 477/939] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:20.330588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc840c42-ed9a-471f-8500-106e2daba416 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.333941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa257ae-df13-4951-9403-62c032051d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.334559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa257ae-df13-4951-9403-62c032051d7a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.334841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa257ae-df13-4951-9403-62c032051d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.335113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa257ae-df13-4951-9403-62c032051d7a None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.335747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa257ae-df13-4951-9403-62c032051d7a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.336051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa257ae-df13-4951-9403-62c032051d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.337441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa257ae-df13-4951-9403-62c032051d7a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.342315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 463/940] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:20.347811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa257ae-df13-4951-9403-62c032051d7a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.429635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.444589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc840c42-ed9a-471f-8500-106e2daba416 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.467741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa257ae-df13-4951-9403-62c032051d7a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.481513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.482112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.482382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.482646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.483403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.484817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.487396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING keystone.server.flask.common [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 00:49:20.503568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2a3d9-fdd4-49cd-98ed-a1014d47affc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.504236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2a3d9-fdd4-49cd-98ed-a1014d47affc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.504452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2a3d9-fdd4-49cd-98ed-a1014d47affc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.504772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2a3d9-fdd4-49cd-98ed-a1014d47affc None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.505681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2a3d9-fdd4-49cd-98ed-a1014d47affc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.506021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2a3d9-fdd4-49cd-98ed-a1014d47affc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.507437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2a3d9-fdd4-49cd-98ed-a1014d47affc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.507543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.sql.core [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 00:49:20.507543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 00:49:20.507543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [parameters: {'id': 'f108d8661cee4581b0cb2397a814a3a1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 00:49:20.507543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81443) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 00:49:20.509165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f108d8661cee4581b0cb2397a814a3a1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f108d8661cee4581b0cb2397a814a3a1. Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:49:20.515118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result.read() Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 00:49:20.516187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application yield resource Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise self.value Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application session.commit() Aug 30 00:49:20.517205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.session.flush() Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application compat.raise_( Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 00:49:20.518730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 00:49:20.519632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application result.read() Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:49:20.520418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [parameters: {'id': 'f108d8661cee4581b0cb2397a814a3a1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.522738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:20.523750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f108d8661cee4581b0cb2397a814a3a1. Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 478/941] 10.222.0.78 () {60 vars in 1140 bytes} [Tue Aug 30 00:49:20 2022] POST /identity/v3/projects => generated 262 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:20.524893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.531165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 464/942] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:20.531871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2a3d9-fdd4-49cd-98ed-a1014d47affc None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.641919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2a3d9-fdd4-49cd-98ed-a1014d47affc None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.643015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526af613-0180-4e68-96b9-0d9beaf125f8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.678697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d380ba-91f0-447b-836f-660ad8d19f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.679404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d380ba-91f0-447b-836f-660ad8d19f41 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.679734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d380ba-91f0-447b-836f-660ad8d19f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.679945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d380ba-91f0-447b-836f-660ad8d19f41 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.680565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d380ba-91f0-447b-836f-660ad8d19f41 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.680663 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12f69c-b2a0-46ef-9c9c-56b64cad41cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.680900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d380ba-91f0-447b-836f-660ad8d19f41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.681248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12f69c-b2a0-46ef-9c9c-56b64cad41cb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.681531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12f69c-b2a0-46ef-9c9c-56b64cad41cb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.681799 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12f69c-b2a0-46ef-9c9c-56b64cad41cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.682489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d380ba-91f0-447b-836f-660ad8d19f41 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.689743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12f69c-b2a0-46ef-9c9c-56b64cad41cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.696050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 479/943] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:20.696176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12f69c-b2a0-46ef-9c9c-56b64cad41cb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.696256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d380ba-91f0-447b-836f-660ad8d19f41 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-1b12f69c-b2a0-46ef-9c9c-56b64cad41cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:20.707820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:20.708756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 465/944] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:20.710028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12f69c-b2a0-46ef-9c9c-56b64cad41cb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.711999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.778419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.813689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d380ba-91f0-447b-836f-660ad8d19f41 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.826417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12f69c-b2a0-46ef-9c9c-56b64cad41cb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.849245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7a25e-2d5b-4ddb-83f9-7fb08bfdae55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.849996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7a25e-2d5b-4ddb-83f9-7fb08bfdae55 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.850277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7a25e-2d5b-4ddb-83f9-7fb08bfdae55 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.850543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7a25e-2d5b-4ddb-83f9-7fb08bfdae55 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.859434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a7a25e-2d5b-4ddb-83f9-7fb08bfdae55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.860856 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a7a25e-2d5b-4ddb-83f9-7fb08bfdae55 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.862586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f5508-b29f-40cc-b42f-bdb8f05effec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.863176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f5508-b29f-40cc-b42f-bdb8f05effec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.863749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f5508-b29f-40cc-b42f-bdb8f05effec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.863749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f5508-b29f-40cc-b42f-bdb8f05effec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.864387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f5508-b29f-40cc-b42f-bdb8f05effec None admin] RBAC: Adding query filter params (interface=public, service_id=0b930e933a144a83870013c78321a6ca, region_id=RegionOne) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.864689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f5508-b29f-40cc-b42f-bdb8f05effec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.866031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f5508-b29f-40cc-b42f-bdb8f05effec None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.871195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 466/945] 10.222.0.78 () {56 vars in 1251 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/endpoints?service_id=0b930e933a144a83870013c78321a6ca&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-f8a7a25e-2d5b-4ddb-83f9-7fb08bfdae55 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:20.872089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:20.873037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:20.875493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 480/946] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:20.877138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f5508-b29f-40cc-b42f-bdb8f05effec None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.881584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7a25e-2d5b-4ddb-83f9-7fb08bfdae55 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.951205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.983849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f5508-b29f-40cc-b42f-bdb8f05effec None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.987989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7a25e-2d5b-4ddb-83f9-7fb08bfdae55 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.003341 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.021959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4aa88a-08cf-430f-aca6-3502af4ba2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.022135 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7b754-5dff-4877-ac5b-de09f236eac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.022558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4aa88a-08cf-430f-aca6-3502af4ba2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.022819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7b754-5dff-4877-ac5b-de09f236eac4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.022951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4aa88a-08cf-430f-aca6-3502af4ba2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.023089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4aa88a-08cf-430f-aca6-3502af4ba2a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.023256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7b754-5dff-4877-ac5b-de09f236eac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.023584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7b754-5dff-4877-ac5b-de09f236eac4 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.023723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4aa88a-08cf-430f-aca6-3502af4ba2a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:21.024020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4aa88a-08cf-430f-aca6-3502af4ba2a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.024348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7b754-5dff-4877-ac5b-de09f236eac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.025459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4aa88a-08cf-430f-aca6-3502af4ba2a1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.025833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7b754-5dff-4877-ac5b-de09f236eac4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.037356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 481/947] 10.222.0.78 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:49:21.037356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7b754-5dff-4877-ac5b-de09f236eac4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.061726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 467/948] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:21.163837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7b754-5dff-4877-ac5b-de09f236eac4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.199687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.207066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54def2b3-f06b-47a1-96ff-214584ede9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.207771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def2b3-f06b-47a1-96ff-214584ede9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.208267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def2b3-f06b-47a1-96ff-214584ede9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.208641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def2b3-f06b-47a1-96ff-214584ede9d2 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.209761 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54def2b3-f06b-47a1-96ff-214584ede9d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:21.210171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54def2b3-f06b-47a1-96ff-214584ede9d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.211750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54def2b3-f06b-47a1-96ff-214584ede9d2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.231378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 482/949] 10.222.0.78 () {56 vars in 1149 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:21.236936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4aa88a-08cf-430f-aca6-3502af4ba2a1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.352460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4aa88a-08cf-430f-aca6-3502af4ba2a1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.388063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc77bd-b9a4-4180-9858-966fba6242b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.388692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc77bd-b9a4-4180-9858-966fba6242b2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.388972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc77bd-b9a4-4180-9858-966fba6242b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.389315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc77bd-b9a4-4180-9858-966fba6242b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.400575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc77bd-b9a4-4180-9858-966fba6242b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.402037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc77bd-b9a4-4180-9858-966fba6242b2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a8fc77bd-b9a4-4180-9858-966fba6242b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:21.417428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:21.418416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:21.419311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:21.419311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:21.419311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:21.419311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 468/950] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:21.424300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54def2b3-f06b-47a1-96ff-214584ede9d2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.529044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54def2b3-f06b-47a1-96ff-214584ede9d2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.565105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0eca-0d54-475e-8e20-5d5e1045c457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.565725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0eca-0d54-475e-8e20-5d5e1045c457 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.566004 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0eca-0d54-475e-8e20-5d5e1045c457 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.566268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0eca-0d54-475e-8e20-5d5e1045c457 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.566905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0eca-0d54-475e-8e20-5d5e1045c457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:21.567204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0eca-0d54-475e-8e20-5d5e1045c457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.568558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0eca-0d54-475e-8e20-5d5e1045c457 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.581478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 483/951] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:21.587040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc77bd-b9a4-4180-9858-966fba6242b2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.618305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.696280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc77bd-b9a4-4180-9858-966fba6242b2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.732478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a05d23-6888-4214-9ef3-91f2fdf87e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.733173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a05d23-6888-4214-9ef3-91f2fdf87e60 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.733505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a05d23-6888-4214-9ef3-91f2fdf87e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.734101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a05d23-6888-4214-9ef3-91f2fdf87e60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.741389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a05d23-6888-4214-9ef3-91f2fdf87e60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.742825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a05d23-6888-4214-9ef3-91f2fdf87e60 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d5a05d23-6888-4214-9ef3-91f2fdf87e60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:21.754232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:21.755167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:21.756203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:21.756203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:21.756203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:21.756203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:21.756203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:21.756203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:21.756203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:21.756203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:21.756203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 469/952] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:21.760990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0eca-0d54-475e-8e20-5d5e1045c457 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.813436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.864610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0eca-0d54-475e-8e20-5d5e1045c457 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.899102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e55b8-c585-4c52-84ee-a2f8b5f22e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.899707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e55b8-c585-4c52-84ee-a2f8b5f22e0a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.899983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e55b8-c585-4c52-84ee-a2f8b5f22e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.900250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e55b8-c585-4c52-84ee-a2f8b5f22e0a None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.900934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e55b8-c585-4c52-84ee-a2f8b5f22e0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:21.901264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e55b8-c585-4c52-84ee-a2f8b5f22e0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.902685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e55b8-c585-4c52-84ee-a2f8b5f22e0a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.916246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 484/953] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:21.922231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a05d23-6888-4214-9ef3-91f2fdf87e60 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.017449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:22.027566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a05d23-6888-4214-9ef3-91f2fdf87e60 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.053273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca990a-8e9f-480c-b5b1-fd3f239dd4c0 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.053598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca990a-8e9f-480c-b5b1-fd3f239dd4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.053815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca990a-8e9f-480c-b5b1-fd3f239dd4c0 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.055433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 485/954] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:22.060236 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ca53-62b2-4f10-9ceb-25e3a480b2ee None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.060522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ca53-62b2-4f10-9ceb-25e3a480b2ee None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.060786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ca53-62b2-4f10-9ceb-25e3a480b2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.066904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22dd60d-f253-4c93-b9f6-e4273d6d35ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.067495 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22dd60d-f253-4c93-b9f6-e4273d6d35ae None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.067772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22dd60d-f253-4c93-b9f6-e4273d6d35ae None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.068043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22dd60d-f253-4c93-b9f6-e4273d6d35ae None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/308bbd2bc1224533b573c80f0565252a/roles/c0568265539e4d40868747758cd54de3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.074321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22dd60d-f253-4c93-b9f6-e4273d6d35ae None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=308bbd2bc1224533b573c80f0565252a, role_id=c0568265539e4d40868747758cd54de3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.075724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22dd60d-f253-4c93-b9f6-e4273d6d35ae None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.085423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cf04ca53-62b2-4f10-9ceb-25e3a480b2ee None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:22.086937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cf04ca53-62b2-4f10-9ceb-25e3a480b2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:22.093111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 470/955] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:21 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/308bbd2bc1224533b573c80f0565252a/roles/c0568265539e4d40868747758cd54de3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:22.190948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 486/956] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:22.471480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22dd60d-f253-4c93-b9f6-e4273d6d35ae None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.582147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22dd60d-f253-4c93-b9f6-e4273d6d35ae None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.617885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabd2fa-b5e5-4844-a2a5-3e53eb543e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.618878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabd2fa-b5e5-4844-a2a5-3e53eb543e7a None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.619364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabd2fa-b5e5-4844-a2a5-3e53eb543e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.619872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabd2fa-b5e5-4844-a2a5-3e53eb543e7a None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.620735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabd2fa-b5e5-4844-a2a5-3e53eb543e7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.622325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabd2fa-b5e5-4844-a2a5-3e53eb543e7a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.644627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 471/957] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:49:22 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:23.117818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0aa4d-86da-4a28-be83-f2cc24acbe51 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.118203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0aa4d-86da-4a28-be83-f2cc24acbe51 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.118337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0aa4d-86da-4a28-be83-f2cc24acbe51 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.119934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 487/958] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:23.126479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec930963-e517-4964-85c6-f40ac35c78c6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.126765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec930963-e517-4964-85c6-f40ac35c78c6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.127027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec930963-e517-4964-85c6-f40ac35c78c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.146167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ec930963-e517-4964-85c6-f40ac35c78c6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:23.147351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ec930963-e517-4964-85c6-f40ac35c78c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:23.150634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:23.233594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 472/959] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:23.267464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886df8b-b69d-4f98-8abc-2de04e36fe7e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.267968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886df8b-b69d-4f98-8abc-2de04e36fe7e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.267968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886df8b-b69d-4f98-8abc-2de04e36fe7e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.269621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 488/960] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:23.274890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934b9e-3c0a-4f49-b3f3-6134781b72fe None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.275181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934b9e-3c0a-4f49-b3f3-6134781b72fe None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.275457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934b9e-3c0a-4f49-b3f3-6134781b72fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.293933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c1934b9e-3c0a-4f49-b3f3-6134781b72fe None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:23.295102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c1934b9e-3c0a-4f49-b3f3-6134781b72fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:23.362042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 473/961] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:23.517763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886df8b-b69d-4f98-8abc-2de04e36fe7e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:23.563496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:23.623547 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886df8b-b69d-4f98-8abc-2de04e36fe7e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:23.660410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe80596-147b-4500-b6fe-e2188685516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:23.661035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe80596-147b-4500-b6fe-e2188685516a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.661335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe80596-147b-4500-b6fe-e2188685516a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.661604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe80596-147b-4500-b6fe-e2188685516a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.662182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe80596-147b-4500-b6fe-e2188685516a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:23.663601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe80596-147b-4500-b6fe-e2188685516a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:23.668532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1934b9e-3c0a-4f49-b3f3-6134781b72fe None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-cfe80596-147b-4500-b6fe-e2188685516a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:23.679116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:23.680555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:23.681746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:23.685241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 489/962] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:23.691020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe80596-147b-4500-b6fe-e2188685516a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:23.748271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:23.789391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1934b9e-3c0a-4f49-b3f3-6134781b72fe None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:23.829959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:23.830824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.831045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe80596-147b-4500-b6fe-e2188685516a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:23.831411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.831855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.832723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:23.833882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:23.835878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:23.838695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WARNING keystone.server.flask.common [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 00:49:23.856245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.sql.core [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 00:49:23.856245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 00:49:23.856245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [parameters: {'user_id': '894071df277b46d0ab34ecabdfed7089', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 00:49:23.856245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81442) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result.read() Aug 30 00:49:23.875512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:49:23.876449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application yield resource Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise self.value Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application session.commit() Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 00:49:23.877438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.session.flush() Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application compat.raise_( Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 00:49:23.878333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:49:23.879219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result.read() Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:49:23.880064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [parameters: {'user_id': '894071df277b46d0ab34ecabdfed7089', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:23.880889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 00:49:23.881692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 00:49:23.883941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 00:49:23.883941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 00:49:23.883941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 00:49:23.883941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 00:49:23.883941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 00:49:23.883941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:23.884714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d79ba5-2368-463d-9409-07557b69f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:23.884714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d79ba5-2368-463d-9409-07557b69f230 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.884714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d79ba5-2368-463d-9409-07557b69f230 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.884714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d79ba5-2368-463d-9409-07557b69f230 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.885291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d79ba5-2368-463d-9409-07557b69f230 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:23.885291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d79ba5-2368-463d-9409-07557b69f230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:23.885291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d79ba5-2368-463d-9409-07557b69f230 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:23.887251 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 474/963] 10.222.0.78 () {60 vars in 1131 bytes} [Tue Aug 30 00:49:23 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:23.889742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 490/964] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:23.895809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:23.897213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d79ba5-2368-463d-9409-07557b69f230 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.008017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d79ba5-2368-463d-9409-07557b69f230 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.040873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3fb7e-f797-43a6-baeb-8890a90c1453 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.051302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da952cd4-e76d-4a29-9c44-31eda4ab3965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.052711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da952cd4-e76d-4a29-9c44-31eda4ab3965 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.053250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da952cd4-e76d-4a29-9c44-31eda4ab3965 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.054162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da952cd4-e76d-4a29-9c44-31eda4ab3965 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.055331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da952cd4-e76d-4a29-9c44-31eda4ab3965 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:24.055868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da952cd4-e76d-4a29-9c44-31eda4ab3965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.058240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da952cd4-e76d-4a29-9c44-31eda4ab3965 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.065747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 491/965] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:24.071859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da952cd4-e76d-4a29-9c44-31eda4ab3965 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.073762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:24.084065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba432e28-2774-408f-9198-97e85e7b4b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.084985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba432e28-2774-408f-9198-97e85e7b4b83 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.085498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba432e28-2774-408f-9198-97e85e7b4b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.085769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba432e28-2774-408f-9198-97e85e7b4b83 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.097876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba432e28-2774-408f-9198-97e85e7b4b83 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.100393 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba432e28-2774-408f-9198-97e85e7b4b83 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.108245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ba432e28-2774-408f-9198-97e85e7b4b83 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:24.116460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:24.117579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:49:24.118678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:24.120620 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 475/966] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:24.122112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11fe02-2bbe-4e40-86ed-04ef582af89a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.122389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11fe02-2bbe-4e40-86ed-04ef582af89a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.122656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11fe02-2bbe-4e40-86ed-04ef582af89a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.127016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 476/967] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:24.127731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11fe02-2bbe-4e40-86ed-04ef582af89a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.191197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da952cd4-e76d-4a29-9c44-31eda4ab3965 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.226896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0255-ebe1-48ff-a69e-7f0096c3a297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.227506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0255-ebe1-48ff-a69e-7f0096c3a297 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.227782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0255-ebe1-48ff-a69e-7f0096c3a297 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.228054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0255-ebe1-48ff-a69e-7f0096c3a297 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.228707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a0255-ebe1-48ff-a69e-7f0096c3a297 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.230109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a0255-ebe1-48ff-a69e-7f0096c3a297 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.241976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11fe02-2bbe-4e40-86ed-04ef582af89a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.265105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 492/968] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:49:24.265105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd936-9a20-4d85-b085-d11b547a2071 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.265105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd936-9a20-4d85-b085-d11b547a2071 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.265105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd936-9a20-4d85-b085-d11b547a2071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.292225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-510dd936-9a20-4d85-b085-d11b547a2071 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:24.293509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-510dd936-9a20-4d85-b085-d11b547a2071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:24.305953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a99e24-599b-4bd5-96ee-02b05cba5b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.307452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a99e24-599b-4bd5-96ee-02b05cba5b22 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.308036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a99e24-599b-4bd5-96ee-02b05cba5b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.308584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a99e24-599b-4bd5-96ee-02b05cba5b22 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.309506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a99e24-599b-4bd5-96ee-02b05cba5b22 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:24.310305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a99e24-599b-4bd5-96ee-02b05cba5b22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.314437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a99e24-599b-4bd5-96ee-02b05cba5b22 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.377108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 477/969] 10.222.0.78 () {56 vars in 1101 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:24.391509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 493/970] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:24.659037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba3de9-ee8b-45e2-940a-755479f475ed None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.659369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba3de9-ee8b-45e2-940a-755479f475ed None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.659590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba3de9-ee8b-45e2-940a-755479f475ed None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.661070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 478/971] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:24.665926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e31876-1dde-4a01-93b9-bb75fe0b43a0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.666218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e31876-1dde-4a01-93b9-bb75fe0b43a0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.666490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e31876-1dde-4a01-93b9-bb75fe0b43a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.677210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba3de9-ee8b-45e2-940a-755479f475ed None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.685231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a1e31876-1dde-4a01-93b9-bb75fe0b43a0 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:24.687358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a1e31876-1dde-4a01-93b9-bb75fe0b43a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:24.702882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:24.726261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:24.766908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 494/972] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:24.786472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba3de9-ee8b-45e2-940a-755479f475ed None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.823076 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251297e-cd4f-49f2-bb03-61a4c4204ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.823690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251297e-cd4f-49f2-bb03-61a4c4204ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.823963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251297e-cd4f-49f2-bb03-61a4c4204ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.824225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251297e-cd4f-49f2-bb03-61a4c4204ce3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.834766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251297e-cd4f-49f2-bb03-61a4c4204ce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-4251297e-cd4f-49f2-bb03-61a4c4204ce3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:24.839441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:24.840369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:24.841283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 479/973] 10.222.0.78 () {56 vars in 1120 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:24.846525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e31876-1dde-4a01-93b9-bb75fe0b43a0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.946524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e31876-1dde-4a01-93b9-bb75fe0b43a0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.980096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fd0e9-6397-4b25-8a0b-8257ce738913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.980690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fd0e9-6397-4b25-8a0b-8257ce738913 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.980962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fd0e9-6397-4b25-8a0b-8257ce738913 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.981261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fd0e9-6397-4b25-8a0b-8257ce738913 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.981908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fd0e9-6397-4b25-8a0b-8257ce738913 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:24.982202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fd0e9-6397-4b25-8a0b-8257ce738913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.984274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fd0e9-6397-4b25-8a0b-8257ce738913 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.992310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 495/974] 10.222.0.78 () {56 vars in 1115 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:24.997713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251297e-cd4f-49f2-bb03-61a4c4204ce3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.040509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fd0e9-6397-4b25-8a0b-8257ce738913 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.053680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.111937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251297e-cd4f-49f2-bb03-61a4c4204ce3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.114285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.150762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74dda5-a150-41a6-a330-d6d558a304f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.151378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dda5-a150-41a6-a330-d6d558a304f6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.151653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dda5-a150-41a6-a330-d6d558a304f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.151918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dda5-a150-41a6-a330-d6d558a304f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.161069 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fd0e9-6397-4b25-8a0b-8257ce738913 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.165911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74dda5-a150-41a6-a330-d6d558a304f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.167345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74dda5-a150-41a6-a330-d6d558a304f6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-fc74dda5-a150-41a6-a330-d6d558a304f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:25.184233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:25.185259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:25.186343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:25.186343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:25.186343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:25.186343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 480/975] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:25.191297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74dda5-a150-41a6-a330-d6d558a304f6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.202105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08a292-6ef9-4d1a-9010-55d6589dcc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.202755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08a292-6ef9-4d1a-9010-55d6589dcc1e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.203036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08a292-6ef9-4d1a-9010-55d6589dcc1e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.203304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08a292-6ef9-4d1a-9010-55d6589dcc1e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.203884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08a292-6ef9-4d1a-9010-55d6589dcc1e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.205271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08a292-6ef9-4d1a-9010-55d6589dcc1e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3e08a292-6ef9-4d1a-9010-55d6589dcc1e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:25.214439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:25.215414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:25.216255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 496/976] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:25.221941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08a292-6ef9-4d1a-9010-55d6589dcc1e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.309946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.312598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74dda5-a150-41a6-a330-d6d558a304f6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.339081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08a292-6ef9-4d1a-9010-55d6589dcc1e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.350390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5befbc-4fde-47e6-a11a-9b94447d960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.350989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5befbc-4fde-47e6-a11a-9b94447d960c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.351263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5befbc-4fde-47e6-a11a-9b94447d960c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.351706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5befbc-4fde-47e6-a11a-9b94447d960c None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.352151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5befbc-4fde-47e6-a11a-9b94447d960c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:25.352455 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5befbc-4fde-47e6-a11a-9b94447d960c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.353822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5befbc-4fde-47e6-a11a-9b94447d960c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.367410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 481/977] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:25.372520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5befbc-4fde-47e6-a11a-9b94447d960c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.379142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb71634-7874-4578-b408-0bfe9c0c28ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.379756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb71634-7874-4578-b408-0bfe9c0c28ed None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.380035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb71634-7874-4578-b408-0bfe9c0c28ed None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.380362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb71634-7874-4578-b408-0bfe9c0c28ed None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.381453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb71634-7874-4578-b408-0bfe9c0c28ed None admin] RBAC: Adding query filter params (name=compute) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:25.381759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb71634-7874-4578-b408-0bfe9c0c28ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.383500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb71634-7874-4578-b408-0bfe9c0c28ed None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.387356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.395332 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 497/978] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:25.396799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb71634-7874-4578-b408-0bfe9c0c28ed None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.484638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5befbc-4fde-47e6-a11a-9b94447d960c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.506192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb71634-7874-4578-b408-0bfe9c0c28ed None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.520896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d04b6-ccb5-4fb0-b184-c78aa9b71a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.521637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d04b6-ccb5-4fb0-b184-c78aa9b71a2b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.521920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d04b6-ccb5-4fb0-b184-c78aa9b71a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.522193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d04b6-ccb5-4fb0-b184-c78aa9b71a2b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.529498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d04b6-ccb5-4fb0-b184-c78aa9b71a2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.530959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d04b6-ccb5-4fb0-b184-c78aa9b71a2b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-983d04b6-ccb5-4fb0-b184-c78aa9b71a2b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:25.543460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:25.544418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:25.546083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 482/979] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:25.547878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad56a-f5f1-4153-b49c-4e34bb3e07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.548702 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad56a-f5f1-4153-b49c-4e34bb3e07a9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.549179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad56a-f5f1-4153-b49c-4e34bb3e07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.549632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad56a-f5f1-4153-b49c-4e34bb3e07a9 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.550429 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad56a-f5f1-4153-b49c-4e34bb3e07a9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:25.550807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d04b6-ccb5-4fb0-b184-c78aa9b71a2b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.551129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad56a-f5f1-4153-b49c-4e34bb3e07a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.552668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad56a-f5f1-4153-b49c-4e34bb3e07a9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.558771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 498/980] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:25.565133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad56a-f5f1-4153-b49c-4e34bb3e07a9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.567915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.638782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.670301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d04b6-ccb5-4fb0-b184-c78aa9b71a2b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.681434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad56a-f5f1-4153-b49c-4e34bb3e07a9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.707634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da486b8-783c-4292-8e6f-dab503bbdafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.708293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da486b8-783c-4292-8e6f-dab503bbdafc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.708567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da486b8-783c-4292-8e6f-dab503bbdafc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.708781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da486b8-783c-4292-8e6f-dab503bbdafc None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.709550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da486b8-783c-4292-8e6f-dab503bbdafc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:25.709860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da486b8-783c-4292-8e6f-dab503bbdafc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.711270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da486b8-783c-4292-8e6f-dab503bbdafc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.717762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75aae4-4a33-435a-9e46-8fedba7f957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.718466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75aae4-4a33-435a-9e46-8fedba7f957b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.718741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75aae4-4a33-435a-9e46-8fedba7f957b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.719007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75aae4-4a33-435a-9e46-8fedba7f957b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.719694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75aae4-4a33-435a-9e46-8fedba7f957b None admin] RBAC: Adding query filter params (interface=public, service_id=9a26f8c4864c465d85e87c3d78fb38f7, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:25.719994 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75aae4-4a33-435a-9e46-8fedba7f957b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.721353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75aae4-4a33-435a-9e46-8fedba7f957b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.726564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 499/981] 10.222.0.78 () {56 vars in 1251 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/endpoints?service_id=9a26f8c4864c465d85e87c3d78fb38f7&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:25.730405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 483/982] 10.222.0.78 () {56 vars in 1149 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:25.732061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75aae4-4a33-435a-9e46-8fedba7f957b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.736420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da486b8-783c-4292-8e6f-dab503bbdafc None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.808859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.842118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da486b8-783c-4292-8e6f-dab503bbdafc None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.848058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75aae4-4a33-435a-9e46-8fedba7f957b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.872974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.879470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7ab5-9eab-4b7d-be06-77db3406610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.880070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7ab5-9eab-4b7d-be06-77db3406610a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.880348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7ab5-9eab-4b7d-be06-77db3406610a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.880621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7ab5-9eab-4b7d-be06-77db3406610a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.885642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56426ce2-52f7-4394-8b97-875a0d4db9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.886247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56426ce2-52f7-4394-8b97-875a0d4db9dc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.886520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56426ce2-52f7-4394-8b97-875a0d4db9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.886780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56426ce2-52f7-4394-8b97-875a0d4db9dc None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.887377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56426ce2-52f7-4394-8b97-875a0d4db9dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.889218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56426ce2-52f7-4394-8b97-875a0d4db9dc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.905957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 500/983] 10.222.0.78 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:49:25.959157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd7ab5-9eab-4b7d-be06-77db3406610a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.960638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd7ab5-9eab-4b7d-be06-77db3406610a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.069991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 484/984] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:49:26.075478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56426ce2-52f7-4394-8b97-875a0d4db9dc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.177682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56426ce2-52f7-4394-8b97-875a0d4db9dc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.207322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.214024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e058575-bc44-4d98-8998-1209b5f29ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.214647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e058575-bc44-4d98-8998-1209b5f29ece None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.214924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e058575-bc44-4d98-8998-1209b5f29ece None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.215191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e058575-bc44-4d98-8998-1209b5f29ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.226925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e058575-bc44-4d98-8998-1209b5f29ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.228364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e058575-bc44-4d98-8998-1209b5f29ece None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3e058575-bc44-4d98-8998-1209b5f29ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:26.242812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:26.243804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:26.244692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:26.244692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:26.244692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:26.246531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 501/985] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:26.250200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7ab5-9eab-4b7d-be06-77db3406610a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.288421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.292492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f703a9f-ffe9-41a1-9811-14b5b6ca3b05 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.292815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f703a9f-ffe9-41a1-9811-14b5b6ca3b05 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.293945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f703a9f-ffe9-41a1-9811-14b5b6ca3b05 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.295566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 502/986] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:26.301378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8e537-fc93-4647-bc3f-25ebf812788e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.301663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8e537-fc93-4647-bc3f-25ebf812788e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.301942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8e537-fc93-4647-bc3f-25ebf812788e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.320843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-26f8e537-fc93-4647-bc3f-25ebf812788e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:26.322107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-26f8e537-fc93-4647-bc3f-25ebf812788e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:26.367904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7ab5-9eab-4b7d-be06-77db3406610a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.394715 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 503/987] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:26.408434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b55321-0be6-4cc0-9039-aecc0b31d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.409044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b55321-0be6-4cc0-9039-aecc0b31d017 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.409356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b55321-0be6-4cc0-9039-aecc0b31d017 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.409625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b55321-0be6-4cc0-9039-aecc0b31d017 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.410270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b55321-0be6-4cc0-9039-aecc0b31d017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:26.410573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b55321-0be6-4cc0-9039-aecc0b31d017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.411920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b55321-0be6-4cc0-9039-aecc0b31d017 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.428179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 485/988] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:26.429434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8e537-fc93-4647-bc3f-25ebf812788e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.531313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8e537-fc93-4647-bc3f-25ebf812788e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.544356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.567814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f914e-069c-4ac0-9098-a696a192a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.568447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f914e-069c-4ac0-9098-a696a192a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.568978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f914e-069c-4ac0-9098-a696a192a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.569287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f914e-069c-4ac0-9098-a696a192a7d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.576079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f914e-069c-4ac0-9098-a696a192a7d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.577653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f914e-069c-4ac0-9098-a696a192a7d3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c09f914e-069c-4ac0-9098-a696a192a7d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:26.589361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:26.590220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:26.591127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:26.591127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:26.591127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:26.591127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:26.591127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:26.591127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:26.591127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:26.591127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:26.593666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 504/989] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:26.596243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b55321-0be6-4cc0-9039-aecc0b31d017 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.670833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f914e-069c-4ac0-9098-a696a192a7d3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.696792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.709206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b55321-0be6-4cc0-9039-aecc0b31d017 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.745317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752bb58-ffb1-46cf-a64b-dad62caff864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.745957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752bb58-ffb1-46cf-a64b-dad62caff864 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.746210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752bb58-ffb1-46cf-a64b-dad62caff864 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.746475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752bb58-ffb1-46cf-a64b-dad62caff864 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.747172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752bb58-ffb1-46cf-a64b-dad62caff864 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:26.747475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752bb58-ffb1-46cf-a64b-dad62caff864 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.748954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752bb58-ffb1-46cf-a64b-dad62caff864 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.766327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 486/990] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:26.768170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752bb58-ffb1-46cf-a64b-dad62caff864 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.793624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f914e-069c-4ac0-9098-a696a192a7d3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.831343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ae059-6e5f-4bac-bec4-620d50512fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.831917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ae059-6e5f-4bac-bec4-620d50512fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.832200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ae059-6e5f-4bac-bec4-620d50512fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.832470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ae059-6e5f-4bac-bec4-620d50512fb0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.833046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ae059-6e5f-4bac-bec4-620d50512fb0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.834444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ae059-6e5f-4bac-bec4-620d50512fb0 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ab0ae059-6e5f-4bac-bec4-620d50512fb0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:26.844001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:26.844993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:26.846495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 505/991] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:26.850978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ae059-6e5f-4bac-bec4-620d50512fb0 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.882685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752bb58-ffb1-46cf-a64b-dad62caff864 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.891310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.924629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c673-ba95-4e0e-ac1f-3e8770cfa1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.925270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c673-ba95-4e0e-ac1f-3e8770cfa1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.925553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c673-ba95-4e0e-ac1f-3e8770cfa1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.925851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c673-ba95-4e0e-ac1f-3e8770cfa1f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.926543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941c673-ba95-4e0e-ac1f-3e8770cfa1f2 None admin] RBAC: Adding query filter params (role.id=c0568265539e4d40868747758cd54de3, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=308bbd2bc1224533b573c80f0565252a) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:26.927149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941c673-ba95-4e0e-ac1f-3e8770cfa1f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.928333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941c673-ba95-4e0e-ac1f-3e8770cfa1f2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.945122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 487/992] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/role_assignments?user.id=308bbd2bc1224533b573c80f0565252a&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=c0568265539e4d40868747758cd54de3 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:26.975144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ae059-6e5f-4bac-bec4-620d50512fb0 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.014061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7f542-21f4-4c21-b40e-aa82b0780d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.015385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f542-21f4-4c21-b40e-aa82b0780d4a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.015725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f542-21f4-4c21-b40e-aa82b0780d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.015996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f542-21f4-4c21-b40e-aa82b0780d4a None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.016757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7f542-21f4-4c21-b40e-aa82b0780d4a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:27.017058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7f542-21f4-4c21-b40e-aa82b0780d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.018474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7f542-21f4-4c21-b40e-aa82b0780d4a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.028474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 506/993] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:27.030431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c673-ba95-4e0e-ac1f-3e8770cfa1f2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.050263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.144997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c673-ba95-4e0e-ac1f-3e8770cfa1f2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.180050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b879ccb-1340-4c43-990e-63fb1431f032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.180658 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b879ccb-1340-4c43-990e-63fb1431f032 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.180925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b879ccb-1340-4c43-990e-63fb1431f032 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.181217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b879ccb-1340-4c43-990e-63fb1431f032 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.181837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b879ccb-1340-4c43-990e-63fb1431f032 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:27.182185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b879ccb-1340-4c43-990e-63fb1431f032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.183527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b879ccb-1340-4c43-990e-63fb1431f032 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.195508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 488/994] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:27.196753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7f542-21f4-4c21-b40e-aa82b0780d4a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.301458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.304867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7f542-21f4-4c21-b40e-aa82b0780d4a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.341265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b54c3-e91c-46b5-b492-aae1493d3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.341937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b54c3-e91c-46b5-b492-aae1493d3a7f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.342183 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b54c3-e91c-46b5-b492-aae1493d3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.342504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b54c3-e91c-46b5-b492-aae1493d3a7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.343206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b54c3-e91c-46b5-b492-aae1493d3a7f None admin] RBAC: Adding query filter params (interface=admin, service_id=0b930e933a144a83870013c78321a6ca, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:27.343513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b54c3-e91c-46b5-b492-aae1493d3a7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.344874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b54c3-e91c-46b5-b492-aae1493d3a7f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.350234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 507/995] 10.222.0.78 () {56 vars in 1249 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/endpoints?service_id=0b930e933a144a83870013c78321a6ca&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:27.355595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b879ccb-1340-4c43-990e-63fb1431f032 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.440820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.462525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b879ccb-1340-4c43-990e-63fb1431f032 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.496212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc04464-eb80-4dac-9bd4-6109a5ef37df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.496859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04464-eb80-4dac-9bd4-6109a5ef37df None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.497157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04464-eb80-4dac-9bd4-6109a5ef37df None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.497425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04464-eb80-4dac-9bd4-6109a5ef37df None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.498020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc04464-eb80-4dac-9bd4-6109a5ef37df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.500018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc04464-eb80-4dac-9bd4-6109a5ef37df None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.513179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 489/996] 10.222.0.78 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:49:27.866061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f8225-f40a-496d-bfd4-979e5330a179 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.866248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f8225-f40a-496d-bfd4-979e5330a179 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.866503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f8225-f40a-496d-bfd4-979e5330a179 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.867991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 508/997] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:27.872964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dbfe2-9bff-4b7d-ac42-ea7c52b7d48b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.873470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dbfe2-9bff-4b7d-ac42-ea7c52b7d48b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.873607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dbfe2-9bff-4b7d-ac42-ea7c52b7d48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.894150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-822dbfe2-9bff-4b7d-ac42-ea7c52b7d48b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:27.895671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-822dbfe2-9bff-4b7d-ac42-ea7c52b7d48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:27.972852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 490/998] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:28.237612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f8225-f40a-496d-bfd4-979e5330a179 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.346459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f8225-f40a-496d-bfd4-979e5330a179 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.381326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a5253-db32-4b7e-b65c-cb4d766ab9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.381942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a5253-db32-4b7e-b65c-cb4d766ab9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.382268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a5253-db32-4b7e-b65c-cb4d766ab9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.382535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a5253-db32-4b7e-b65c-cb4d766ab9a6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.383105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a5253-db32-4b7e-b65c-cb4d766ab9a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.384456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a5253-db32-4b7e-b65c-cb4d766ab9a6 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d28a5253-db32-4b7e-b65c-cb4d766ab9a6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:28.394517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:28.395510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:49:28.396367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:28.400608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 509/999] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:28.401867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dbfe2-9bff-4b7d-ac42-ea7c52b7d48b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.479878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:28.507584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dbfe2-9bff-4b7d-ac42-ea7c52b7d48b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.542780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebfe9e-b0ff-40c4-8a65-ed15f8605166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.543562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebfe9e-b0ff-40c4-8a65-ed15f8605166 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.543935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebfe9e-b0ff-40c4-8a65-ed15f8605166 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.544303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebfe9e-b0ff-40c4-8a65-ed15f8605166 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.545026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebfe9e-b0ff-40c4-8a65-ed15f8605166 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:28.545624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebfe9e-b0ff-40c4-8a65-ed15f8605166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.546983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebfe9e-b0ff-40c4-8a65-ed15f8605166 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.556594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 491/1000] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:28.557578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a5253-db32-4b7e-b65c-cb4d766ab9a6 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.587115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:28.662423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a5253-db32-4b7e-b65c-cb4d766ab9a6 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.696682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d38236-f9b9-433b-a5c7-bb91b05737e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.697317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d38236-f9b9-433b-a5c7-bb91b05737e3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.697596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d38236-f9b9-433b-a5c7-bb91b05737e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.697866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d38236-f9b9-433b-a5c7-bb91b05737e3 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.698485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d38236-f9b9-433b-a5c7-bb91b05737e3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:28.698784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d38236-f9b9-433b-a5c7-bb91b05737e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.700139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d38236-f9b9-433b-a5c7-bb91b05737e3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.710137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 510/1001] 10.222.0.78 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:28.711405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebfe9e-b0ff-40c4-8a65-ed15f8605166 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.816082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebfe9e-b0ff-40c4-8a65-ed15f8605166 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.850534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784fe20-7142-4d62-ac35-6d7d37974f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.851134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784fe20-7142-4d62-ac35-6d7d37974f8d None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.851408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784fe20-7142-4d62-ac35-6d7d37974f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.851677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784fe20-7142-4d62-ac35-6d7d37974f8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.852351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784fe20-7142-4d62-ac35-6d7d37974f8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.853732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784fe20-7142-4d62-ac35-6d7d37974f8d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.862984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:28.876840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 492/1002] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:28 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:28.934303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4893fd-d108-44b1-8126-a991c8d4f4b2 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.934625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4893fd-d108-44b1-8126-a991c8d4f4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.934858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4893fd-d108-44b1-8126-a991c8d4f4b2 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.937117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 511/1003] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:28.942826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e2ed3-dc7e-4e46-831d-ccf67d280445 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.943145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e2ed3-dc7e-4e46-831d-ccf67d280445 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.943390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e2ed3-dc7e-4e46-831d-ccf67d280445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.963163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ba2e2ed3-dc7e-4e46-831d-ccf67d280445 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:28.964384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ba2e2ed3-dc7e-4e46-831d-ccf67d280445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:29.049010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 493/1004] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:28 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:29.321514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4893fd-d108-44b1-8126-a991c8d4f4b2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.418288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.430141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4893fd-d108-44b1-8126-a991c8d4f4b2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.465124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550fafd-7388-4224-a69a-9eb5d8f23fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.465748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550fafd-7388-4224-a69a-9eb5d8f23fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.466022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550fafd-7388-4224-a69a-9eb5d8f23fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.466283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550fafd-7388-4224-a69a-9eb5d8f23fa4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.466853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550fafd-7388-4224-a69a-9eb5d8f23fa4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.468312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550fafd-7388-4224-a69a-9eb5d8f23fa4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.473172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982a4ee-570e-41b8-ba15-dbf0d15676b3 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.473466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982a4ee-570e-41b8-ba15-dbf0d15676b3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.473746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982a4ee-570e-41b8-ba15-dbf0d15676b3 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.475281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 494/1005] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-5550fafd-7388-4224-a69a-9eb5d8f23fa4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:29.477324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:29.478267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:29.479941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 512/1006] 10.222.0.78 () {56 vars in 1105 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:29.480733 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34618365-c508-45e2-8211-a0c129a743e3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.480733 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34618365-c508-45e2-8211-a0c129a743e3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.480733 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34618365-c508-45e2-8211-a0c129a743e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.486004 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550fafd-7388-4224-a69a-9eb5d8f23fa4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.500149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-34618365-c508-45e2-8211-a0c129a743e3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:29.501381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-34618365-c508-45e2-8211-a0c129a743e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:29.583485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 495/1007] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:29 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:29.590339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550fafd-7388-4224-a69a-9eb5d8f23fa4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.624721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4c3da-f647-4e62-99ff-4aeb3069fbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.625366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4c3da-f647-4e62-99ff-4aeb3069fbbb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.625642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4c3da-f647-4e62-99ff-4aeb3069fbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.625906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4c3da-f647-4e62-99ff-4aeb3069fbbb None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.626526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4c3da-f647-4e62-99ff-4aeb3069fbbb None admin] RBAC: Adding query filter params (name=image) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:29.626823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4c3da-f647-4e62-99ff-4aeb3069fbbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.628169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4c3da-f647-4e62-99ff-4aeb3069fbbb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.637541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 513/1008] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:29.639802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34618365-c508-45e2-8211-a0c129a743e3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.694636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.747087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34618365-c508-45e2-8211-a0c129a743e3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.782538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5c60d-32a9-41ce-93b6-316d40b43132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.783139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5c60d-32a9-41ce-93b6-316d40b43132 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.783415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5c60d-32a9-41ce-93b6-316d40b43132 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.783683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5c60d-32a9-41ce-93b6-316d40b43132 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.784303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5c60d-32a9-41ce-93b6-316d40b43132 None admin] RBAC: Adding query filter params (type=image) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:29.784605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5c60d-32a9-41ce-93b6-316d40b43132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.786055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5c60d-32a9-41ce-93b6-316d40b43132 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.797331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 496/1009] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:29.870350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4c3da-f647-4e62-99ff-4aeb3069fbbb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.896414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.990099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4c3da-f647-4e62-99ff-4aeb3069fbbb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.024254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da4247-ad92-4546-9380-ee1fbce4cffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.024882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da4247-ad92-4546-9380-ee1fbce4cffb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.025192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da4247-ad92-4546-9380-ee1fbce4cffb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.025466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da4247-ad92-4546-9380-ee1fbce4cffb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.026045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da4247-ad92-4546-9380-ee1fbce4cffb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:30.027407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da4247-ad92-4546-9380-ee1fbce4cffb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-a3da4247-ad92-4546-9380-ee1fbce4cffb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:30.036934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:30.038026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:30.038883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 514/1010] 10.222.0.78 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:30.046350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5c60d-32a9-41ce-93b6-316d40b43132 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:30.155916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5c60d-32a9-41ce-93b6-316d40b43132 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.169787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:30.192224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8bb78-4d0e-4a5d-995a-e54424ef9ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.192827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8bb78-4d0e-4a5d-995a-e54424ef9ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.193130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8bb78-4d0e-4a5d-995a-e54424ef9ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.193421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8bb78-4d0e-4a5d-995a-e54424ef9ab1 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.194281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8bb78-4d0e-4a5d-995a-e54424ef9ab1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:30.194634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8bb78-4d0e-4a5d-995a-e54424ef9ab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:30.195989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8bb78-4d0e-4a5d-995a-e54424ef9ab1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:30.205139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 497/1011] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:30.207815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da4247-ad92-4546-9380-ee1fbce4cffb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:30.299248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:30.317530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da4247-ad92-4546-9380-ee1fbce4cffb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.353206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df504837-ed9f-4126-8439-2414e0f70be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.353807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504837-ed9f-4126-8439-2414e0f70be9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.354082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504837-ed9f-4126-8439-2414e0f70be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.354347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504837-ed9f-4126-8439-2414e0f70be9 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.354960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df504837-ed9f-4126-8439-2414e0f70be9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:30.355252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df504837-ed9f-4126-8439-2414e0f70be9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:30.356598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df504837-ed9f-4126-8439-2414e0f70be9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:30.366628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 515/1012] 10.222.0.78 () {56 vars in 1122 bytes} [Tue Aug 30 00:49:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:30.367717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8bb78-4d0e-4a5d-995a-e54424ef9ab1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:30.468557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8bb78-4d0e-4a5d-995a-e54424ef9ab1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.503446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c2580-0c05-46c3-b6ab-3d424b3a5c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.504044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c2580-0c05-46c3-b6ab-3d424b3a5c02 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.504325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c2580-0c05-46c3-b6ab-3d424b3a5c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.504598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c2580-0c05-46c3-b6ab-3d424b3a5c02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.505282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c2580-0c05-46c3-b6ab-3d424b3a5c02 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:30.506621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c2580-0c05-46c3-b6ab-3d424b3a5c02 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:30.529122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 498/1013] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:30.919260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50c7b2-6857-496c-9a6c-19a92edd0049 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.919818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50c7b2-6857-496c-9a6c-19a92edd0049 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.919818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50c7b2-6857-496c-9a6c-19a92edd0049 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.922309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 516/1014] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:30.927408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf1de6-4175-44d8-81c6-5c9b73fdfe3a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.927704 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf1de6-4175-44d8-81c6-5c9b73fdfe3a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.927981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf1de6-4175-44d8-81c6-5c9b73fdfe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.947281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0cbf1de6-4175-44d8-81c6-5c9b73fdfe3a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:30.948463 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0cbf1de6-4175-44d8-81c6-5c9b73fdfe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:30.979769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.026584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 499/1015] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:30 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:31.305858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50c7b2-6857-496c-9a6c-19a92edd0049 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.409820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50c7b2-6857-496c-9a6c-19a92edd0049 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.445900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a99b87-8f91-49f2-999b-c7333da84e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.446512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99b87-8f91-49f2-999b-c7333da84e3f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.446787 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99b87-8f91-49f2-999b-c7333da84e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.447055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99b87-8f91-49f2-999b-c7333da84e3f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.457393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a99b87-8f91-49f2-999b-c7333da84e3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-c1a99b87-8f91-49f2-999b-c7333da84e3f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:31.462162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:31.463087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:31.463908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:31.465156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 517/1016] 10.222.0.78 () {56 vars in 1120 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:31.469802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf1de6-4175-44d8-81c6-5c9b73fdfe3a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.572205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf1de6-4175-44d8-81c6-5c9b73fdfe3a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.609403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2eca8-de3b-4be9-bd61-42b625ad6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.610011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2eca8-de3b-4be9-bd61-42b625ad6c7a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.610288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2eca8-de3b-4be9-bd61-42b625ad6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.610562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2eca8-de3b-4be9-bd61-42b625ad6c7a None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.611231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2eca8-de3b-4be9-bd61-42b625ad6c7a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:31.611536 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2eca8-de3b-4be9-bd61-42b625ad6c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.612888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2eca8-de3b-4be9-bd61-42b625ad6c7a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.625988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 500/1017] 10.222.0.78 () {56 vars in 1115 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:31.627415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a99b87-8f91-49f2-999b-c7333da84e3f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.640043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.722376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a7a89-57d0-458e-bddd-e5cf5aa7041a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.722707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a7a89-57d0-458e-bddd-e5cf5aa7041a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.722917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a7a89-57d0-458e-bddd-e5cf5aa7041a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.724383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 501/1018] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:31.729009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dab1f-a746-4d6f-9b62-b54faa8fdb96 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.729325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dab1f-a746-4d6f-9b62-b54faa8fdb96 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.729600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dab1f-a746-4d6f-9b62-b54faa8fdb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.736776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a99b87-8f91-49f2-999b-c7333da84e3f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.750032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-248dab1f-a746-4d6f-9b62-b54faa8fdb96 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:31.751189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-248dab1f-a746-4d6f-9b62-b54faa8fdb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:31.771969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.774860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc6c02-a586-4ece-9e19-776a849023bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.775483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc6c02-a586-4ece-9e19-776a849023bb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.775795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc6c02-a586-4ece-9e19-776a849023bb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.776037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc6c02-a586-4ece-9e19-776a849023bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.786572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc6c02-a586-4ece-9e19-776a849023bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.788950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc6c02-a586-4ece-9e19-776a849023bb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-95dc6c02-a586-4ece-9e19-776a849023bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:31.805143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:31.806096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:31.807005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:31.807005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:31.807005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:31.807005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 518/1019] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:31.812923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc6c02-a586-4ece-9e19-776a849023bb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.822876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 502/1020] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:31 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:31.872011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.920719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc6c02-a586-4ece-9e19-776a849023bb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.955549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853642ee-5341-4828-abb0-3589d0fb99ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.956149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853642ee-5341-4828-abb0-3589d0fb99ff None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.956426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853642ee-5341-4828-abb0-3589d0fb99ff None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.956750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853642ee-5341-4828-abb0-3589d0fb99ff None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.957414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853642ee-5341-4828-abb0-3589d0fb99ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:31.957715 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853642ee-5341-4828-abb0-3589d0fb99ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.959077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853642ee-5341-4828-abb0-3589d0fb99ff None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.975674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 519/1021] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:31.976982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248dab1f-a746-4d6f-9b62-b54faa8fdb96 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.079609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248dab1f-a746-4d6f-9b62-b54faa8fdb96 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.096328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853642ee-5341-4828-abb0-3589d0fb99ff None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.115702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54db3b8-4983-4d44-9227-736feb3aeaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.116335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54db3b8-4983-4d44-9227-736feb3aeaca None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.116614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54db3b8-4983-4d44-9227-736feb3aeaca None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.116987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54db3b8-4983-4d44-9227-736feb3aeaca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.124882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54db3b8-4983-4d44-9227-736feb3aeaca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.126348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54db3b8-4983-4d44-9227-736feb3aeaca None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d54db3b8-4983-4d44-9227-736feb3aeaca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:32.139272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:32.140263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:32.141280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 503/1022] 10.222.0.78 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:32.146039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54db3b8-4983-4d44-9227-736feb3aeaca None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.164387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.208238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853642ee-5341-4828-abb0-3589d0fb99ff None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.229473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.250244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3c932-5e0b-4aa1-8e15-7c0e4b99dc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.250851 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3c932-5e0b-4aa1-8e15-7c0e4b99dc04 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.251125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3c932-5e0b-4aa1-8e15-7c0e4b99dc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.251389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3c932-5e0b-4aa1-8e15-7c0e4b99dc04 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.252030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3c932-5e0b-4aa1-8e15-7c0e4b99dc04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.253440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3c932-5e0b-4aa1-8e15-7c0e4b99dc04 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.258850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54db3b8-4983-4d44-9227-736feb3aeaca None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.275240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 520/1023] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:49:32 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:32.307362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f92ce3-1136-429c-9dd7-375b8891a72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.307985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92ce3-1136-429c-9dd7-375b8891a72a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.308264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92ce3-1136-429c-9dd7-375b8891a72a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.308533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92ce3-1136-429c-9dd7-375b8891a72a None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.309338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f92ce3-1136-429c-9dd7-375b8891a72a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:32.309649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f92ce3-1136-429c-9dd7-375b8891a72a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.312559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f92ce3-1136-429c-9dd7-375b8891a72a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.334296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 504/1024] 10.222.0.78 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:32.341116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3c932-5e0b-4aa1-8e15-7c0e4b99dc04 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.452861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3c932-5e0b-4aa1-8e15-7c0e4b99dc04 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.488933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae68707-8fff-4082-9721-bbef13021b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.489576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68707-8fff-4082-9721-bbef13021b6c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.489852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68707-8fff-4082-9721-bbef13021b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.490118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68707-8fff-4082-9721-bbef13021b6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.495340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03440d89-6035-4031-a74b-0a013b5c42dc None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.495800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03440d89-6035-4031-a74b-0a013b5c42dc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.495877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03440d89-6035-4031-a74b-0a013b5c42dc None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.497399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 505/1025] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:32.502883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51116331-61c0-4cf1-ae6d-195542e112c1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.503169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51116331-61c0-4cf1-ae6d-195542e112c1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.503448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51116331-61c0-4cf1-ae6d-195542e112c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.522667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-51116331-61c0-4cf1-ae6d-195542e112c1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:32.523942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-51116331-61c0-4cf1-ae6d-195542e112c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:32.547571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.570975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae68707-8fff-4082-9721-bbef13021b6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.572610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae68707-8fff-4082-9721-bbef13021b6c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.592737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.598767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 506/1026] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:32.680443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 521/1027] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:32.686123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51116331-61c0-4cf1-ae6d-195542e112c1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.788322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51116331-61c0-4cf1-ae6d-195542e112c1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.822392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e9c21-6f55-4ded-878e-9cfac8e34621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.823012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e9c21-6f55-4ded-878e-9cfac8e34621 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.823290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e9c21-6f55-4ded-878e-9cfac8e34621 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.823557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e9c21-6f55-4ded-878e-9cfac8e34621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.834060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e9c21-6f55-4ded-878e-9cfac8e34621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.835484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e9c21-6f55-4ded-878e-9cfac8e34621 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-fd2e9c21-6f55-4ded-878e-9cfac8e34621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:32.850457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:32.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:32.852306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:32.852306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:32.852306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:32.853423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 507/1028] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:32.857047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae68707-8fff-4082-9721-bbef13021b6c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.874465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e9c21-6f55-4ded-878e-9cfac8e34621 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.928359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.941689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.966685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae68707-8fff-4082-9721-bbef13021b6c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.984752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e9c21-6f55-4ded-878e-9cfac8e34621 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.006063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839355-ea96-46c3-8169-a9c3a560142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.006657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839355-ea96-46c3-8169-a9c3a560142a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.006972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839355-ea96-46c3-8169-a9c3a560142a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.007263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839355-ea96-46c3-8169-a9c3a560142a None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.008128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839355-ea96-46c3-8169-a9c3a560142a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:33.008306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839355-ea96-46c3-8169-a9c3a560142a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.009723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839355-ea96-46c3-8169-a9c3a560142a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.018965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf5050-3fef-47c9-84e3-0cd661aa5e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.019589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf5050-3fef-47c9-84e3-0cd661aa5e9a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.019858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf5050-3fef-47c9-84e3-0cd661aa5e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.020124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf5050-3fef-47c9-84e3-0cd661aa5e9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.021242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf5050-3fef-47c9-84e3-0cd661aa5e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.022648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf5050-3fef-47c9-84e3-0cd661aa5e9a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.029128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 508/1029] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:33.029253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 522/1030] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:33.030442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf5050-3fef-47c9-84e3-0cd661aa5e9a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.031152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839355-ea96-46c3-8169-a9c3a560142a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.129751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839355-ea96-46c3-8169-a9c3a560142a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.136104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf5050-3fef-47c9-84e3-0cd661aa5e9a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.157298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:33.166859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35027df9-8a61-4318-9140-de01c8d9b1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.167717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35027df9-8a61-4318-9140-de01c8d9b1bf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.168059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35027df9-8a61-4318-9140-de01c8d9b1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.168376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35027df9-8a61-4318-9140-de01c8d9b1bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.172548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd2be-0b36-4692-a777-4aab90652512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.173175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd2be-0b36-4692-a777-4aab90652512 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.173450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd2be-0b36-4692-a777-4aab90652512 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.173716 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd2be-0b36-4692-a777-4aab90652512 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.174351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd2be-0b36-4692-a777-4aab90652512 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.175490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35027df9-8a61-4318-9140-de01c8d9b1bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.175743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd2be-0b36-4692-a777-4aab90652512 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.176971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35027df9-8a61-4318-9140-de01c8d9b1bf None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.179668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-35027df9-8a61-4318-9140-de01c8d9b1bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:33.190995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:33.192099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:33.193020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:33.193020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:33.193020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:33.193020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:33.193020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:33.193020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:33.193020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:33.193020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:33.193020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 523/1031] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:33.198541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35027df9-8a61-4318-9140-de01c8d9b1bf None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.213119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 509/1032] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:49:33 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:49:33.320407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35027df9-8a61-4318-9140-de01c8d9b1bf None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.361484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024130c4-edae-426f-b374-be8095698512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.362143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024130c4-edae-426f-b374-be8095698512 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.362351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024130c4-edae-426f-b374-be8095698512 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.363393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024130c4-edae-426f-b374-be8095698512 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.363393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024130c4-edae-426f-b374-be8095698512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:33.363581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024130c4-edae-426f-b374-be8095698512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.364983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024130c4-edae-426f-b374-be8095698512 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.377902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 524/1033] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:33.384147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd2be-0b36-4692-a777-4aab90652512 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.491223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd2be-0b36-4692-a777-4aab90652512 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.526520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dca1b6-0337-4b2f-936a-4021994cc405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.527154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca1b6-0337-4b2f-936a-4021994cc405 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.527400 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca1b6-0337-4b2f-936a-4021994cc405 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.527673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca1b6-0337-4b2f-936a-4021994cc405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.528401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dca1b6-0337-4b2f-936a-4021994cc405 None admin] RBAC: Adding query filter params (role.id=c0568265539e4d40868747758cd54de3, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=f9b88c495fcb47a289a6a45276c891e0) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:33.528707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dca1b6-0337-4b2f-936a-4021994cc405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.530283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dca1b6-0337-4b2f-936a-4021994cc405 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.541102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 510/1034] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/role_assignments?user.id=f9b88c495fcb47a289a6a45276c891e0&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=c0568265539e4d40868747758cd54de3 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:34.242367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd65df-e980-4318-b79e-a9c8ebeae73a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.242367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd65df-e980-4318-b79e-a9c8ebeae73a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.242367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd65df-e980-4318-b79e-a9c8ebeae73a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.243212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 525/1035] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:34.248068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b140d2-b531-45ae-9805-e0674ea56a88 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.249005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b140d2-b531-45ae-9805-e0674ea56a88 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.249302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b140d2-b531-45ae-9805-e0674ea56a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.270944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e1b140d2-b531-45ae-9805-e0674ea56a88 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:34.272130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e1b140d2-b531-45ae-9805-e0674ea56a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:34.301403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.342106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 511/1036] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:34.617614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd65df-e980-4318-b79e-a9c8ebeae73a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.678092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.721086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd65df-e980-4318-b79e-a9c8ebeae73a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.754669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fce314-83a8-4286-bdcb-f3937bbc70fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.755286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fce314-83a8-4286-bdcb-f3937bbc70fc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.755559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fce314-83a8-4286-bdcb-f3937bbc70fc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.755823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fce314-83a8-4286-bdcb-f3937bbc70fc None admin] PATH_INFO: `/v3/services/image` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.756393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fce314-83a8-4286-bdcb-f3937bbc70fc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.757779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fce314-83a8-4286-bdcb-f3937bbc70fc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-f7fce314-83a8-4286-bdcb-f3937bbc70fc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:34.767022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:34.768084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:34.768958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 526/1037] 10.222.0.78 () {56 vars in 1105 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:34.773968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b140d2-b531-45ae-9805-e0674ea56a88 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.872127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b140d2-b531-45ae-9805-e0674ea56a88 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.906781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb72e6-5fa4-4f72-a5e2-42b78b57979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.907367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb72e6-5fa4-4f72-a5e2-42b78b57979c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.907643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb72e6-5fa4-4f72-a5e2-42b78b57979c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.907908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb72e6-5fa4-4f72-a5e2-42b78b57979c None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.908531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb72e6-5fa4-4f72-a5e2-42b78b57979c None admin] RBAC: Adding query filter params (name=image) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:34.908829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb72e6-5fa4-4f72-a5e2-42b78b57979c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.910202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb72e6-5fa4-4f72-a5e2-42b78b57979c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.919553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 512/1038] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:34.921357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fce314-83a8-4286-bdcb-f3937bbc70fc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.024572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fce314-83a8-4286-bdcb-f3937bbc70fc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.048336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.059888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ff8b6-4563-4af6-88e6-f852f00645d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.060492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff8b6-4563-4af6-88e6-f852f00645d1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.060917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff8b6-4563-4af6-88e6-f852f00645d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.061708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff8b6-4563-4af6-88e6-f852f00645d1 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.062328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ff8b6-4563-4af6-88e6-f852f00645d1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:35.062629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ff8b6-4563-4af6-88e6-f852f00645d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.063986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ff8b6-4563-4af6-88e6-f852f00645d1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.074073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 527/1039] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:35.075555 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb72e6-5fa4-4f72-a5e2-42b78b57979c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.111708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.183686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb72e6-5fa4-4f72-a5e2-42b78b57979c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.196893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c9482-1ed5-42f0-b728-fc238dbbd191 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.197199 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c9482-1ed5-42f0-b728-fc238dbbd191 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.197470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c9482-1ed5-42f0-b728-fc238dbbd191 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.199055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 528/1040] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:35.204758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f13c9d-b7d9-498c-9f53-3f1c073d4dac None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.205038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f13c9d-b7d9-498c-9f53-3f1c073d4dac None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.205331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f13c9d-b7d9-498c-9f53-3f1c073d4dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.221200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d098f-45be-4f55-a1a5-39c1c5506ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.221807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d098f-45be-4f55-a1a5-39c1c5506ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.222082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d098f-45be-4f55-a1a5-39c1c5506ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.222345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d098f-45be-4f55-a1a5-39c1c5506ad2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.223037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d098f-45be-4f55-a1a5-39c1c5506ad2 None admin] RBAC: Adding query filter params (interface=public, service_id=c8a204d936014cd681b3e72eb83e2907, region_id=RegionOne) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:35.223340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d098f-45be-4f55-a1a5-39c1c5506ad2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.224677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d098f-45be-4f55-a1a5-39c1c5506ad2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.227812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-00f13c9d-b7d9-498c-9f53-3f1c073d4dac None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:35.229292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-00f13c9d-b7d9-498c-9f53-3f1c073d4dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:35.230141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 513/1041] 10.222.0.78 () {56 vars in 1251 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/endpoints?service_id=c8a204d936014cd681b3e72eb83e2907&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:35.236195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d098f-45be-4f55-a1a5-39c1c5506ad2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.306182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 529/1042] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:35.339777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.347358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d098f-45be-4f55-a1a5-39c1c5506ad2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.383160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789d67d-4f97-4cb4-a4a4-c1baa0c4ddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.383763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789d67d-4f97-4cb4-a4a4-c1baa0c4ddc8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.384042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789d67d-4f97-4cb4-a4a4-c1baa0c4ddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.384313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789d67d-4f97-4cb4-a4a4-c1baa0c4ddc8 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.384918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789d67d-4f97-4cb4-a4a4-c1baa0c4ddc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.386300 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789d67d-4f97-4cb4-a4a4-c1baa0c4ddc8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.401374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 514/1043] 10.222.0.78 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:49:35.486386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41111fa-e8d2-4b92-9f4d-06e061c700a9 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.486579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41111fa-e8d2-4b92-9f4d-06e061c700a9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.486841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41111fa-e8d2-4b92-9f4d-06e061c700a9 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.488363 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 530/1044] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:35.493752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5e43d-f712-4221-adfa-cdf5afc5bdd1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.494044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5e43d-f712-4221-adfa-cdf5afc5bdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.494322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5e43d-f712-4221-adfa-cdf5afc5bdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.514722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-dbb5e43d-f712-4221-adfa-cdf5afc5bdd1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:35.515872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-dbb5e43d-f712-4221-adfa-cdf5afc5bdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:35.574857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41111fa-e8d2-4b92-9f4d-06e061c700a9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.592501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 515/1045] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:35.681644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41111fa-e8d2-4b92-9f4d-06e061c700a9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.700508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.728725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc7ef5-6a10-414f-b906-e249303c1a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.729387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc7ef5-6a10-414f-b906-e249303c1a01 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.729668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc7ef5-6a10-414f-b906-e249303c1a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.729938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc7ef5-6a10-414f-b906-e249303c1a01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.731061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc7ef5-6a10-414f-b906-e249303c1a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.732502 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc7ef5-6a10-414f-b906-e249303c1a01 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.734857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 531/1046] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:35.740394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5e43d-f712-4221-adfa-cdf5afc5bdd1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.828097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.842138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5e43d-f712-4221-adfa-cdf5afc5bdd1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.866726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc7ef5-6a10-414f-b906-e249303c1a01 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.879007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0490f51-ba8a-4d28-bb29-3edc067441f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.879620 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0490f51-ba8a-4d28-bb29-3edc067441f1 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.879882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0490f51-ba8a-4d28-bb29-3edc067441f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.880145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0490f51-ba8a-4d28-bb29-3edc067441f1 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.880793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0490f51-ba8a-4d28-bb29-3edc067441f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.882282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0490f51-ba8a-4d28-bb29-3edc067441f1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.907059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 516/1047] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:49:35 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:35.986723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc7ef5-6a10-414f-b906-e249303c1a01 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.023184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbe143-942e-4067-8f3f-f32e297d221a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.023736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbe143-942e-4067-8f3f-f32e297d221a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.024011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbe143-942e-4067-8f3f-f32e297d221a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.024274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbe143-942e-4067-8f3f-f32e297d221a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.036997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbe143-942e-4067-8f3f-f32e297d221a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4dfbe143-942e-4067-8f3f-f32e297d221a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:36.041869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:36.042811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:36.043660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:36.045966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 532/1048] 10.222.0.78 () {56 vars in 1120 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:36.049640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0490f51-ba8a-4d28-bb29-3edc067441f1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.167215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0490f51-ba8a-4d28-bb29-3edc067441f1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.201345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b9435-1765-492f-bae7-70972325ec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.202638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b9435-1765-492f-bae7-70972325ec27 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.202913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b9435-1765-492f-bae7-70972325ec27 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.203178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b9435-1765-492f-bae7-70972325ec27 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.203835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b9435-1765-492f-bae7-70972325ec27 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:36.204133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b9435-1765-492f-bae7-70972325ec27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.205498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b9435-1765-492f-bae7-70972325ec27 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.213627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 517/1049] 10.222.0.78 () {56 vars in 1115 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:36.219622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbe143-942e-4067-8f3f-f32e297d221a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.279440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.335187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbe143-942e-4067-8f3f-f32e297d221a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.368526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7748e-e7ae-48de-ac7e-6aa36d86b3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.369173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7748e-e7ae-48de-ac7e-6aa36d86b3bf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.369453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7748e-e7ae-48de-ac7e-6aa36d86b3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.369719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7748e-e7ae-48de-ac7e-6aa36d86b3bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.381090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7748e-e7ae-48de-ac7e-6aa36d86b3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.382538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7748e-e7ae-48de-ac7e-6aa36d86b3bf None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-6fe7748e-e7ae-48de-ac7e-6aa36d86b3bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:36.397345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:36.398309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:36.399192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:36.399192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:36.399192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:36.399192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 533/1050] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:36.404388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b9435-1765-492f-bae7-70972325ec27 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.414005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.512743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b9435-1765-492f-bae7-70972325ec27 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.549123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae105714-7522-47bd-8225-19754054af5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.549645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae105714-7522-47bd-8225-19754054af5e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.549917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae105714-7522-47bd-8225-19754054af5e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.550186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae105714-7522-47bd-8225-19754054af5e None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.550816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae105714-7522-47bd-8225-19754054af5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:36.551165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae105714-7522-47bd-8225-19754054af5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.552545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae105714-7522-47bd-8225-19754054af5e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.569564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 518/1051] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:36.571134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7748e-e7ae-48de-ac7e-6aa36d86b3bf None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.665936 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.671013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7748e-e7ae-48de-ac7e-6aa36d86b3bf None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.705653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2e02d-dc99-4a57-98c2-e399c25c5080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.706269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2e02d-dc99-4a57-98c2-e399c25c5080 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.706543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2e02d-dc99-4a57-98c2-e399c25c5080 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.706857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2e02d-dc99-4a57-98c2-e399c25c5080 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.713953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2e02d-dc99-4a57-98c2-e399c25c5080 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.715364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2e02d-dc99-4a57-98c2-e399c25c5080 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-f9f2e02d-dc99-4a57-98c2-e399c25c5080 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:36.726405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:36.727334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:36.728275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 534/1052] 10.222.0.78 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:36.734166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae105714-7522-47bd-8225-19754054af5e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.798992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.834929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae105714-7522-47bd-8225-19754054af5e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.868340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74061f-f5bf-4de7-aad8-fe9913c9697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.868936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74061f-f5bf-4de7-aad8-fe9913c9697b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.869255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74061f-f5bf-4de7-aad8-fe9913c9697b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.869523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74061f-f5bf-4de7-aad8-fe9913c9697b None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.870239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74061f-f5bf-4de7-aad8-fe9913c9697b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:36.870537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74061f-f5bf-4de7-aad8-fe9913c9697b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.871930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74061f-f5bf-4de7-aad8-fe9913c9697b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.889771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 519/1053] 10.222.0.78 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:36.895115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2e02d-dc99-4a57-98c2-e399c25c5080 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.000743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2e02d-dc99-4a57-98c2-e399c25c5080 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.034485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402f520-91d9-4b5f-aaed-3007f64382b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.035111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402f520-91d9-4b5f-aaed-3007f64382b8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.035384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402f520-91d9-4b5f-aaed-3007f64382b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.035652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402f520-91d9-4b5f-aaed-3007f64382b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.046275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402f520-91d9-4b5f-aaed-3007f64382b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.047717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402f520-91d9-4b5f-aaed-3007f64382b8 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-1402f520-91d9-4b5f-aaed-3007f64382b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:37.062401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:37.063391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:37.064294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:37.064294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:37.064294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:37.064294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 535/1054] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:37.070886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74061f-f5bf-4de7-aad8-fe9913c9697b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.174990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74061f-f5bf-4de7-aad8-fe9913c9697b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.198403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.210304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216f4a3-1db4-41eb-94a4-aea5dac9bfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.210909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216f4a3-1db4-41eb-94a4-aea5dac9bfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.211184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216f4a3-1db4-41eb-94a4-aea5dac9bfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.211455 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216f4a3-1db4-41eb-94a4-aea5dac9bfc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.212090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216f4a3-1db4-41eb-94a4-aea5dac9bfc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:37.212387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216f4a3-1db4-41eb-94a4-aea5dac9bfc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.213767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216f4a3-1db4-41eb-94a4-aea5dac9bfc6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.230362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 520/1055] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:37.231730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402f520-91d9-4b5f-aaed-3007f64382b8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.245743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.341542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402f520-91d9-4b5f-aaed-3007f64382b8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.362952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57192d89-15f1-4710-a41b-1a2d9b9581ba None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.363280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57192d89-15f1-4710-a41b-1a2d9b9581ba None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.363502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57192d89-15f1-4710-a41b-1a2d9b9581ba None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.365032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 521/1056] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:37.369560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7881450-9913-42f5-a5a8-c1d45c0e7540 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.369837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7881450-9913-42f5-a5a8-c1d45c0e7540 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.370110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7881450-9913-42f5-a5a8-c1d45c0e7540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.378538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da036d41-4552-4d67-985f-54767cf5834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.379181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da036d41-4552-4d67-985f-54767cf5834b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.379465 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da036d41-4552-4d67-985f-54767cf5834b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.379735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da036d41-4552-4d67-985f-54767cf5834b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.386974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da036d41-4552-4d67-985f-54767cf5834b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.388424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da036d41-4552-4d67-985f-54767cf5834b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.389199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b7881450-9913-42f5-a5a8-c1d45c0e7540 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:37.390377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b7881450-9913-42f5-a5a8-c1d45c0e7540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-da036d41-4552-4d67-985f-54767cf5834b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:37.399911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:37.400789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:37.401738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:37.401738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:37.401738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:37.401738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:37.401738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:37.401738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:37.401738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:37.401738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:37.405416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 536/1057] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:37.407063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da036d41-4552-4d67-985f-54767cf5834b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.469375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 522/1058] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:37.476811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.510665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da036d41-4552-4d67-985f-54767cf5834b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.543822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207999fc-2694-49f6-b0db-7cd9ddc35e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.544426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207999fc-2694-49f6-b0db-7cd9ddc35e82 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.544704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207999fc-2694-49f6-b0db-7cd9ddc35e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.545596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207999fc-2694-49f6-b0db-7cd9ddc35e82 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.546285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207999fc-2694-49f6-b0db-7cd9ddc35e82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:37.546591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207999fc-2694-49f6-b0db-7cd9ddc35e82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.548012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207999fc-2694-49f6-b0db-7cd9ddc35e82 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.560171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 537/1059] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:37.566642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7881450-9913-42f5-a5a8-c1d45c0e7540 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.687731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7881450-9913-42f5-a5a8-c1d45c0e7540 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.703920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.725044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02809f16-fa5e-4831-a1db-c2cdcf970398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.725712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02809f16-fa5e-4831-a1db-c2cdcf970398 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.725985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02809f16-fa5e-4831-a1db-c2cdcf970398 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.726251 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02809f16-fa5e-4831-a1db-c2cdcf970398 None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/f9b88c495fcb47a289a6a45276c891e0/roles/c0568265539e4d40868747758cd54de3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.728382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02809f16-fa5e-4831-a1db-c2cdcf970398 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=f9b88c495fcb47a289a6a45276c891e0, role_id=c0568265539e4d40868747758cd54de3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.729820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02809f16-fa5e-4831-a1db-c2cdcf970398 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.738918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207999fc-2694-49f6-b0db-7cd9ddc35e82 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.744582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 523/1060] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:37 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/f9b88c495fcb47a289a6a45276c891e0/roles/c0568265539e4d40868747758cd54de3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:37.852172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207999fc-2694-49f6-b0db-7cd9ddc35e82 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.885617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.894456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49011c-59c3-4aa0-8b1b-1f9403f497c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.895444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49011c-59c3-4aa0-8b1b-1f9403f497c4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.895925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49011c-59c3-4aa0-8b1b-1f9403f497c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.896395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49011c-59c3-4aa0-8b1b-1f9403f497c4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.897216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49011c-59c3-4aa0-8b1b-1f9403f497c4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.898789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49011c-59c3-4aa0-8b1b-1f9403f497c4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-eb49011c-59c3-4aa0-8b1b-1f9403f497c4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:37.908422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:37.909513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:37.910404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 538/1061] 10.222.0.78 () {56 vars in 1105 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:37.917311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02809f16-fa5e-4831-a1db-c2cdcf970398 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.936673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e360032-7918-44be-b642-badc9e3a76fa None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.937234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e360032-7918-44be-b642-badc9e3a76fa None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.937519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e360032-7918-44be-b642-badc9e3a76fa None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.939419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 539/1062] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:37.944716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12dd59-c2d3-46b3-9233-d36f278a811a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.945003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12dd59-c2d3-46b3-9233-d36f278a811a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.945339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12dd59-c2d3-46b3-9233-d36f278a811a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.964073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3d12dd59-c2d3-46b3-9233-d36f278a811a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:37.965294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3d12dd59-c2d3-46b3-9233-d36f278a811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:38.026244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02809f16-fa5e-4831-a1db-c2cdcf970398 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.036706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 540/1063] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:38.061998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5698a4-6d00-41f2-a3b6-173a84510fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.062606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5698a4-6d00-41f2-a3b6-173a84510fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.062882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5698a4-6d00-41f2-a3b6-173a84510fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.063151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5698a4-6d00-41f2-a3b6-173a84510fa4 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.063778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5698a4-6d00-41f2-a3b6-173a84510fa4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:38.064078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5698a4-6d00-41f2-a3b6-173a84510fa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.065460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5698a4-6d00-41f2-a3b6-173a84510fa4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.070775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 524/1064] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:38.076323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12dd59-c2d3-46b3-9233-d36f278a811a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.176309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12dd59-c2d3-46b3-9233-d36f278a811a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.202855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.214510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52234db4-b2de-4fb6-a18b-3018014174fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.215230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52234db4-b2de-4fb6-a18b-3018014174fd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.215612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52234db4-b2de-4fb6-a18b-3018014174fd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.215983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52234db4-b2de-4fb6-a18b-3018014174fd None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.216710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52234db4-b2de-4fb6-a18b-3018014174fd None admin] RBAC: Adding query filter params (type=image) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:38.217306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52234db4-b2de-4fb6-a18b-3018014174fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.218917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52234db4-b2de-4fb6-a18b-3018014174fd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.228652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 541/1065] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:38.229845 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5698a4-6d00-41f2-a3b6-173a84510fa4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.276067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.314661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52234db4-b2de-4fb6-a18b-3018014174fd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.342934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5698a4-6d00-41f2-a3b6-173a84510fa4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.384455 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d4b39-5b96-4209-8801-30e2cfb9ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.385687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d4b39-5b96-4209-8801-30e2cfb9ed7a None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.385978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d4b39-5b96-4209-8801-30e2cfb9ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.386253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d4b39-5b96-4209-8801-30e2cfb9ed7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.387213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d4b39-5b96-4209-8801-30e2cfb9ed7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.389533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d4b39-5b96-4209-8801-30e2cfb9ed7a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.412214 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 525/1066] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:38.427551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52234db4-b2de-4fb6-a18b-3018014174fd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.474693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d55043-c918-48d6-9250-dc581178b923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.475336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d55043-c918-48d6-9250-dc581178b923 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.475622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d55043-c918-48d6-9250-dc581178b923 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.475892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d55043-c918-48d6-9250-dc581178b923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.489582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d55043-c918-48d6-9250-dc581178b923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-26d55043-c918-48d6-9250-dc581178b923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:38.494425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:38.495386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:38.496474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 542/1067] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:38.502453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d4b39-5b96-4209-8801-30e2cfb9ed7a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.610071 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.618066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d4b39-5b96-4209-8801-30e2cfb9ed7a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.654671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04c0cf-648a-4230-91b3-4043302562e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.655280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04c0cf-648a-4230-91b3-4043302562e7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.655556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04c0cf-648a-4230-91b3-4043302562e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.655825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04c0cf-648a-4230-91b3-4043302562e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.656496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04c0cf-648a-4230-91b3-4043302562e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:38.656793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04c0cf-648a-4230-91b3-4043302562e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.658183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04c0cf-648a-4230-91b3-4043302562e7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.667840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 526/1068] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:38.674141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d55043-c918-48d6-9250-dc581178b923 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.722639 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.779959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d55043-c918-48d6-9250-dc581178b923 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.814452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015a514-c500-4adc-8998-42e288996a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.815079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015a514-c500-4adc-8998-42e288996a3f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.815363 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015a514-c500-4adc-8998-42e288996a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.815635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015a514-c500-4adc-8998-42e288996a3f None admin] PATH_INFO: `/v3/users/0b18297eab844ec19c7b4f770888e044` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.829646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015a514-c500-4adc-8998-42e288996a3f None admin] RBAC: Authorizing `identity:get_user(user_id=0b18297eab844ec19c7b4f770888e044)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.831130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015a514-c500-4adc-8998-42e288996a3f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.832909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 543/1069] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/users/0b18297eab844ec19c7b4f770888e044 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:38.839229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04c0cf-648a-4230-91b3-4043302562e7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.943849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04c0cf-648a-4230-91b3-4043302562e7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.978218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0acc-4be4-4cc7-9332-b7d7a7a599f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.978814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0acc-4be4-4cc7-9332-b7d7a7a599f3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.979088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0acc-4be4-4cc7-9332-b7d7a7a599f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.979386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0acc-4be4-4cc7-9332-b7d7a7a599f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.003855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:39.048474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0acc-4be4-4cc7-9332-b7d7a7a599f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:39.049926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0acc-4be4-4cc7-9332-b7d7a7a599f3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:39.152464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 527/1070] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:39.154151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015a514-c500-4adc-8998-42e288996a3f None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:39.267395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015a514-c500-4adc-8998-42e288996a3f None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:39.268971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:39.301955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3712a-4cfb-480f-98f3-23b542185f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:39.302583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3712a-4cfb-480f-98f3-23b542185f71 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.302858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3712a-4cfb-480f-98f3-23b542185f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.303124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3712a-4cfb-480f-98f3-23b542185f71 None admin] PATH_INFO: `/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.304238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3712a-4cfb-480f-98f3-23b542185f71 None admin] RBAC: Authorizing `identity:get_project(project_id=50ba627cb1e140bdafd9e390ff06ba3c)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:39.305692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3712a-4cfb-480f-98f3-23b542185f71 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:39.307532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 544/1071] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:39 2022] GET /identity/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:39.313979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0acc-4be4-4cc7-9332-b7d7a7a599f3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:39.370717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:39.415730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0acc-4be4-4cc7-9332-b7d7a7a599f3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:39.450976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f839464-1056-4497-ad66-6c726939cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:39.452116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f839464-1056-4497-ad66-6c726939cfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.452387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f839464-1056-4497-ad66-6c726939cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.452651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f839464-1056-4497-ad66-6c726939cfe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.453398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f839464-1056-4497-ad66-6c726939cfe6 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=50ba627cb1e140bdafd9e390ff06ba3c, user.id=0b18297eab844ec19c7b4f770888e044) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:39.453702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f839464-1056-4497-ad66-6c726939cfe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:39.455107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f839464-1056-4497-ad66-6c726939cfe6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:39.460753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 528/1072] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:39 2022] GET /identity/v3/role_assignments?user.id=0b18297eab844ec19c7b4f770888e044&scope.project.id=50ba627cb1e140bdafd9e390ff06ba3c&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:39.702858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc30b2-026c-487e-90ce-acf69cf771ff None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.703413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc30b2-026c-487e-90ce-acf69cf771ff None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.703413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc30b2-026c-487e-90ce-acf69cf771ff None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.704905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 545/1073] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:39.710053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cd288-0b24-459c-848e-afd1ba26c224 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.710342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cd288-0b24-459c-848e-afd1ba26c224 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.710614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cd288-0b24-459c-848e-afd1ba26c224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.729516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-db6cd288-0b24-459c-848e-afd1ba26c224 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:39.730700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-db6cd288-0b24-459c-848e-afd1ba26c224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:39.799210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 529/1074] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:40.069232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc30b2-026c-487e-90ce-acf69cf771ff None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.170078 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc30b2-026c-487e-90ce-acf69cf771ff None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.205512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f4b7a-1dbc-4ce7-8294-90610fb186cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.206126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4b7a-1dbc-4ce7-8294-90610fb186cd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.206403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4b7a-1dbc-4ce7-8294-90610fb186cd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.206670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4b7a-1dbc-4ce7-8294-90610fb186cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.216659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f4b7a-1dbc-4ce7-8294-90610fb186cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ad5f4b7a-1dbc-4ce7-8294-90610fb186cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:40.221814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:40.222739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:49:40.223602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:40.225460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 546/1075] 10.222.0.78 () {56 vars in 1120 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:40.228925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cd288-0b24-459c-848e-afd1ba26c224 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.290284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:40.344194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cd288-0b24-459c-848e-afd1ba26c224 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.379106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2d663-35d5-4c9b-a7bf-ebbcd29b1a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.379717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2d663-35d5-4c9b-a7bf-ebbcd29b1a0b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.379827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd659ae-2ed6-48ba-bdb3-dc2d80034a7e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.380013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2d663-35d5-4c9b-a7bf-ebbcd29b1a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.380129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd659ae-2ed6-48ba-bdb3-dc2d80034a7e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.380296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2d663-35d5-4c9b-a7bf-ebbcd29b1a0b None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.380392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd659ae-2ed6-48ba-bdb3-dc2d80034a7e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.380918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2d663-35d5-4c9b-a7bf-ebbcd29b1a0b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:40.381255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2d663-35d5-4c9b-a7bf-ebbcd29b1a0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.381822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 547/1076] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:40.382748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2d663-35d5-4c9b-a7bf-ebbcd29b1a0b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:40.386454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4600361-6d15-4991-895c-89735aff7275 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.386731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4600361-6d15-4991-895c-89735aff7275 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.387000 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4600361-6d15-4991-895c-89735aff7275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.392322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 530/1077] 10.222.0.78 () {56 vars in 1115 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:40.398373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2d663-35d5-4c9b-a7bf-ebbcd29b1a0b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.408183 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b4600361-6d15-4991-895c-89735aff7275 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:40.409382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b4600361-6d15-4991-895c-89735aff7275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:40.436473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:40.485152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 548/1078] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:40.508911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2d663-35d5-4c9b-a7bf-ebbcd29b1a0b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.524389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:40.549299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02bbf-7b11-45b9-a23c-eb7d10696213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.549925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02bbf-7b11-45b9-a23c-eb7d10696213 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.550204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02bbf-7b11-45b9-a23c-eb7d10696213 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.550475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02bbf-7b11-45b9-a23c-eb7d10696213 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.561423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d02bbf-7b11-45b9-a23c-eb7d10696213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.562913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d02bbf-7b11-45b9-a23c-eb7d10696213 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-65d02bbf-7b11-45b9-a23c-eb7d10696213 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:40.578161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:40.579210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:40.580643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:40.580643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:40.580643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:40.580643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 531/1079] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:40.585272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4600361-6d15-4991-895c-89735aff7275 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.690342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4600361-6d15-4991-895c-89735aff7275 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.724693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad927ef4-7527-459c-b126-c5e43a3d3130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.725338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad927ef4-7527-459c-b126-c5e43a3d3130 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.725604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad927ef4-7527-459c-b126-c5e43a3d3130 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.725868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad927ef4-7527-459c-b126-c5e43a3d3130 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.726504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad927ef4-7527-459c-b126-c5e43a3d3130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:40.726805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad927ef4-7527-459c-b126-c5e43a3d3130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.728159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad927ef4-7527-459c-b126-c5e43a3d3130 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:40.744069 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 549/1080] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:40.745323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02bbf-7b11-45b9-a23c-eb7d10696213 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.750582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad927ef4-7527-459c-b126-c5e43a3d3130 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.774798 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:40.852756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02bbf-7b11-45b9-a23c-eb7d10696213 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.856766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad927ef4-7527-459c-b126-c5e43a3d3130 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.889556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ea24e-3c14-41ea-86aa-f01ecf24ece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.890303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ea24e-3c14-41ea-86aa-f01ecf24ece7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.890653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ea24e-3c14-41ea-86aa-f01ecf24ece7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.890932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ea24e-3c14-41ea-86aa-f01ecf24ece7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.893188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7017ba3-f530-40cf-a0d2-018b5aceb26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.894317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7017ba3-f530-40cf-a0d2-018b5aceb26e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.894594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7017ba3-f530-40cf-a0d2-018b5aceb26e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.894858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7017ba3-f530-40cf-a0d2-018b5aceb26e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.898769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ea24e-3c14-41ea-86aa-f01ecf24ece7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.900193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ea24e-3c14-41ea-86aa-f01ecf24ece7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ef8ea24e-3c14-41ea-86aa-f01ecf24ece7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:40.911654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:40.917604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 532/1081] 10.222.0.78 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:40.918623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ea24e-3c14-41ea-86aa-f01ecf24ece7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.947924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:40.975480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7017ba3-f530-40cf-a0d2-018b5aceb26e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.976930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7017ba3-f530-40cf-a0d2-018b5aceb26e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:40.999830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.030313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ea24e-3c14-41ea-86aa-f01ecf24ece7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.064619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89e99b-7974-409e-8859-af050e0310a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.065256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89e99b-7974-409e-8859-af050e0310a5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.065538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89e99b-7974-409e-8859-af050e0310a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.065815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89e99b-7974-409e-8859-af050e0310a5 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.066537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89e99b-7974-409e-8859-af050e0310a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:41.066843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89e99b-7974-409e-8859-af050e0310a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.068257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89e99b-7974-409e-8859-af050e0310a5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.085970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 533/1082] 10.222.0.78 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:41.086967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 550/1083] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:49:41.091757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89e99b-7974-409e-8859-af050e0310a5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.092994 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7017ba3-f530-40cf-a0d2-018b5aceb26e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.174895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.195970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89e99b-7974-409e-8859-af050e0310a5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.198781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7017ba3-f530-40cf-a0d2-018b5aceb26e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.220765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.233108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa69dc6-8c64-4557-a0a3-e3f15002c71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.233867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa69dc6-8c64-4557-a0a3-e3f15002c71b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.234147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa69dc6-8c64-4557-a0a3-e3f15002c71b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.234417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa69dc6-8c64-4557-a0a3-e3f15002c71b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.237897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c61b00-2b79-4601-8dc8-675a3ad04357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.238530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c61b00-2b79-4601-8dc8-675a3ad04357 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.238813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c61b00-2b79-4601-8dc8-675a3ad04357 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.239086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c61b00-2b79-4601-8dc8-675a3ad04357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.255476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c61b00-2b79-4601-8dc8-675a3ad04357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.257057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c61b00-2b79-4601-8dc8-675a3ad04357 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e4c61b00-2b79-4601-8dc8-675a3ad04357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:41.273479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:41.274484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:41.275379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:41.275379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:41.275379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:41.275379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 551/1084] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:41.281212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c61b00-2b79-4601-8dc8-675a3ad04357 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.314780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa69dc6-8c64-4557-a0a3-e3f15002c71b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.316206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa69dc6-8c64-4557-a0a3-e3f15002c71b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.386950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c61b00-2b79-4601-8dc8-675a3ad04357 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.397436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.418945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 534/1085] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:49:41.421646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e17f3-dee6-48c5-8987-fdf0ea2b1a3a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.421931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e17f3-dee6-48c5-8987-fdf0ea2b1a3a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.422200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e17f3-dee6-48c5-8987-fdf0ea2b1a3a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.423458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 535/1086] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:41.425268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb648fb-0f46-49ec-8711-3d5495c41b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.425418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e17f3-dee6-48c5-8987-fdf0ea2b1a3a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.425876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb648fb-0f46-49ec-8711-3d5495c41b1b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.426150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb648fb-0f46-49ec-8711-3d5495c41b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.426413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb648fb-0f46-49ec-8711-3d5495c41b1b None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.427047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb648fb-0f46-49ec-8711-3d5495c41b1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:41.427422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb648fb-0f46-49ec-8711-3d5495c41b1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.428782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb648fb-0f46-49ec-8711-3d5495c41b1b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.440886 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 552/1087] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:41.442154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c0633-fd55-4456-891e-7a0c8a45db5e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.442437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c0633-fd55-4456-891e-7a0c8a45db5e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.442704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c0633-fd55-4456-891e-7a0c8a45db5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.464085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9d5c0633-fd55-4456-891e-7a0c8a45db5e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:41.465786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9d5c0633-fd55-4456-891e-7a0c8a45db5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:41.500638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.550774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 553/1088] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:41.551837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e17f3-dee6-48c5-8987-fdf0ea2b1a3a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.591388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0bc10-cb57-4386-b099-b419ec087def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.593249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0bc10-cb57-4386-b099-b419ec087def None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.593521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0bc10-cb57-4386-b099-b419ec087def None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.593879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0bc10-cb57-4386-b099-b419ec087def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.604662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0bc10-cb57-4386-b099-b419ec087def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.606426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0bc10-cb57-4386-b099-b419ec087def None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-36f0bc10-cb57-4386-b099-b419ec087def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:41.622595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:41.624117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:41.625094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:41.625094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:41.625094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:41.625094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 536/1089] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:41.630292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c0633-fd55-4456-891e-7a0c8a45db5e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.739699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c0633-fd55-4456-891e-7a0c8a45db5e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.774384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87199f-cfcb-4ec3-ab1d-4576f210e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.774984 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87199f-cfcb-4ec3-ab1d-4576f210e002 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.775258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87199f-cfcb-4ec3-ab1d-4576f210e002 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.775532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87199f-cfcb-4ec3-ab1d-4576f210e002 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.776159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87199f-cfcb-4ec3-ab1d-4576f210e002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:41.776455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87199f-cfcb-4ec3-ab1d-4576f210e002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.777832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87199f-cfcb-4ec3-ab1d-4576f210e002 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.788652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 554/1090] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:41.796073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0bc10-cb57-4386-b099-b419ec087def None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.821679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87199f-cfcb-4ec3-ab1d-4576f210e002 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.830556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.846429 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.896969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0bc10-cb57-4386-b099-b419ec087def None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.921863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87199f-cfcb-4ec3-ab1d-4576f210e002 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.929988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0e350-6b75-4d30-9f11-6e487a540ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.930614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0e350-6b75-4d30-9f11-6e487a540ebb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.930894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0e350-6b75-4d30-9f11-6e487a540ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.931163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0e350-6b75-4d30-9f11-6e487a540ebb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.938453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0e350-6b75-4d30-9f11-6e487a540ebb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.939882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0e350-6b75-4d30-9f11-6e487a540ebb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-e4d0e350-6b75-4d30-9f11-6e487a540ebb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:41.951468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:41.952543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:41.953712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:41.953712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:41.953712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:41.953712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:41.953712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:41.953712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:41.953712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:41.953712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:41.953712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 537/1091] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:41.958564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0e350-6b75-4d30-9f11-6e487a540ebb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.959152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972263cf-7d4a-4a76-86c7-c8fb8aabc106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.959771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972263cf-7d4a-4a76-86c7-c8fb8aabc106 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.960043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972263cf-7d4a-4a76-86c7-c8fb8aabc106 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.960307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972263cf-7d4a-4a76-86c7-c8fb8aabc106 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.970799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972263cf-7d4a-4a76-86c7-c8fb8aabc106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-972263cf-7d4a-4a76-86c7-c8fb8aabc106 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:41.975537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:41.976479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:49:41.977387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 555/1092] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:41.982897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972263cf-7d4a-4a76-86c7-c8fb8aabc106 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.051519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.059083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0e350-6b75-4d30-9f11-6e487a540ebb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.063959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.082318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972263cf-7d4a-4a76-86c7-c8fb8aabc106 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.093309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04576144-dcaf-4b5a-ba10-9316bf7af791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.093911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04576144-dcaf-4b5a-ba10-9316bf7af791 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.094191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04576144-dcaf-4b5a-ba10-9316bf7af791 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.094467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04576144-dcaf-4b5a-ba10-9316bf7af791 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.095183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04576144-dcaf-4b5a-ba10-9316bf7af791 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:42.095491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04576144-dcaf-4b5a-ba10-9316bf7af791 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.096947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04576144-dcaf-4b5a-ba10-9316bf7af791 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.109865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 538/1093] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:42.115811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04576144-dcaf-4b5a-ba10-9316bf7af791 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.116261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f30e5-e000-4b27-bf76-626c5694a02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.116862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f30e5-e000-4b27-bf76-626c5694a02b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.117169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f30e5-e000-4b27-bf76-626c5694a02b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.117443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f30e5-e000-4b27-bf76-626c5694a02b None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.118102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f30e5-e000-4b27-bf76-626c5694a02b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:42.118405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f30e5-e000-4b27-bf76-626c5694a02b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.119776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f30e5-e000-4b27-bf76-626c5694a02b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.129036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 556/1094] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:42.134612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f30e5-e000-4b27-bf76-626c5694a02b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.215451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04576144-dcaf-4b5a-ba10-9316bf7af791 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.233251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f30e5-e000-4b27-bf76-626c5694a02b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.251369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84102b5b-3c9e-49af-9001-c515a0fb9637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.251977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84102b5b-3c9e-49af-9001-c515a0fb9637 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.252256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84102b5b-3c9e-49af-9001-c515a0fb9637 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.252529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84102b5b-3c9e-49af-9001-c515a0fb9637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.253745 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84102b5b-3c9e-49af-9001-c515a0fb9637 None admin] RBAC: Adding query filter params (role.id=c0568265539e4d40868747758cd54de3, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=f9b88c495fcb47a289a6a45276c891e0) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:42.254059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84102b5b-3c9e-49af-9001-c515a0fb9637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.255498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84102b5b-3c9e-49af-9001-c515a0fb9637 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.261897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 539/1095] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/role_assignments?user.id=f9b88c495fcb47a289a6a45276c891e0&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=c0568265539e4d40868747758cd54de3 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:42.270852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa01a7-2a30-405c-ae41-d3604297fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.271499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa01a7-2a30-405c-ae41-d3604297fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.271774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa01a7-2a30-405c-ae41-d3604297fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.272042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa01a7-2a30-405c-ae41-d3604297fbc4 None admin] PATH_INFO: `/v3/users/0b18297eab844ec19c7b4f770888e044` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.273289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa01a7-2a30-405c-ae41-d3604297fbc4 None admin] RBAC: Authorizing `identity:get_user(user_id=0b18297eab844ec19c7b4f770888e044)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.274747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa01a7-2a30-405c-ae41-d3604297fbc4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.276416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 557/1096] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/users/0b18297eab844ec19c7b4f770888e044 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:42.282554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84102b5b-3c9e-49af-9001-c515a0fb9637 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.302350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.389280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84102b5b-3c9e-49af-9001-c515a0fb9637 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.427560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cbf41-3d40-4c3f-97ad-3462f65081bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.428180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cbf41-3d40-4c3f-97ad-3462f65081bc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.428453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cbf41-3d40-4c3f-97ad-3462f65081bc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.428719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cbf41-3d40-4c3f-97ad-3462f65081bc None admin] PATH_INFO: `/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.429859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cbf41-3d40-4c3f-97ad-3462f65081bc None admin] RBAC: Authorizing `identity:get_project(project_id=50ba627cb1e140bdafd9e390ff06ba3c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.431286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cbf41-3d40-4c3f-97ad-3462f65081bc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.433095 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 540/1097] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:42.440213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa01a7-2a30-405c-ae41-d3604297fbc4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.456441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.543592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa01a7-2a30-405c-ae41-d3604297fbc4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.579830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f074e-fff6-4aa0-a043-77239551fd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.580406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f074e-fff6-4aa0-a043-77239551fd13 None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.580682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f074e-fff6-4aa0-a043-77239551fd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.580948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f074e-fff6-4aa0-a043-77239551fd13 None admin] PATH_INFO: `/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c/users/0b18297eab844ec19c7b4f770888e044/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.582859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f074e-fff6-4aa0-a043-77239551fd13 None admin] RBAC: Authorizing `identity:create_grant(project_id=50ba627cb1e140bdafd9e390ff06ba3c, user_id=0b18297eab844ec19c7b4f770888e044, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.584306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f074e-fff6-4aa0-a043-77239551fd13 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.598725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 558/1098] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:42 2022] PUT /identity/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c/users/0b18297eab844ec19c7b4f770888e044/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:44.521504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118a6da-382e-485b-9000-c0f407c502d1 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.522004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118a6da-382e-485b-9000-c0f407c502d1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.522004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118a6da-382e-485b-9000-c0f407c502d1 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.523305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 541/1099] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:44.528402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd40f3-85be-405a-8a98-046b18613f91 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.528800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd40f3-85be-405a-8a98-046b18613f91 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.528945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd40f3-85be-405a-8a98-046b18613f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.547901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9fbd40f3-85be-405a-8a98-046b18613f91 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:44.549056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9fbd40f3-85be-405a-8a98-046b18613f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:44.612763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 559/1100] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:44.877227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118a6da-382e-485b-9000-c0f407c502d1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:44.960278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:44.976720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118a6da-382e-485b-9000-c0f407c502d1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.008806 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dee7c-98ad-41e6-a5a4-29be9346c88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.009438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dee7c-98ad-41e6-a5a4-29be9346c88b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.009714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dee7c-98ad-41e6-a5a4-29be9346c88b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.009980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dee7c-98ad-41e6-a5a4-29be9346c88b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.020293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dee7c-98ad-41e6-a5a4-29be9346c88b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ea0dee7c-98ad-41e6-a5a4-29be9346c88b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:45.024935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:45.026121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:49:45.027137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 542/1101] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:45.032794 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd40f3-85be-405a-8a98-046b18613f91 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.034994 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.130487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd40f3-85be-405a-8a98-046b18613f91 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.164449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19150c-1bc4-438f-abcc-85c9500a3bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.165047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19150c-1bc4-438f-abcc-85c9500a3bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.165350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19150c-1bc4-438f-abcc-85c9500a3bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.165617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19150c-1bc4-438f-abcc-85c9500a3bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.166271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19150c-1bc4-438f-abcc-85c9500a3bc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:45.166569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19150c-1bc4-438f-abcc-85c9500a3bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.167920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19150c-1bc4-438f-abcc-85c9500a3bc2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.176230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 560/1102] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:45.182001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dee7c-98ad-41e6-a5a4-29be9346c88b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.281988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dee7c-98ad-41e6-a5a4-29be9346c88b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.315889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca5f3-6af3-45ea-8533-b56339a18285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.316505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca5f3-6af3-45ea-8533-b56339a18285 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.316783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca5f3-6af3-45ea-8533-b56339a18285 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.317052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca5f3-6af3-45ea-8533-b56339a18285 None admin] PATH_INFO: `/v3/users/0b18297eab844ec19c7b4f770888e044` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.318190 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dca5f3-6af3-45ea-8533-b56339a18285 None admin] RBAC: Authorizing `identity:get_user(user_id=0b18297eab844ec19c7b4f770888e044)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.319862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dca5f3-6af3-45ea-8533-b56339a18285 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.321461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 543/1103] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/users/0b18297eab844ec19c7b4f770888e044 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:45.328031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19150c-1bc4-438f-abcc-85c9500a3bc2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.388447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.428128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19150c-1bc4-438f-abcc-85c9500a3bc2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.461246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279be9e-2ffa-4805-836e-a82e7f47b888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.461846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279be9e-2ffa-4805-836e-a82e7f47b888 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.462125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279be9e-2ffa-4805-836e-a82e7f47b888 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.462395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279be9e-2ffa-4805-836e-a82e7f47b888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.528264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279be9e-2ffa-4805-836e-a82e7f47b888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.529827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279be9e-2ffa-4805-836e-a82e7f47b888 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.587793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.625696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 561/1104] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:45.631616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca5f3-6af3-45ea-8533-b56339a18285 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.647211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.730245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca5f3-6af3-45ea-8533-b56339a18285 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.763852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040c5b4-efdf-4bd0-a678-90bf87cadf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.764842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040c5b4-efdf-4bd0-a678-90bf87cadf99 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.765150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040c5b4-efdf-4bd0-a678-90bf87cadf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.765427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040c5b4-efdf-4bd0-a678-90bf87cadf99 None admin] PATH_INFO: `/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.766557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040c5b4-efdf-4bd0-a678-90bf87cadf99 None admin] RBAC: Authorizing `identity:get_project(project_id=50ba627cb1e140bdafd9e390ff06ba3c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.767969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040c5b4-efdf-4bd0-a678-90bf87cadf99 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.769642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 544/1105] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:45.775780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279be9e-2ffa-4805-836e-a82e7f47b888 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.872229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279be9e-2ffa-4805-836e-a82e7f47b888 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.905888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4a07b-cb11-4db8-b97c-d1f9a626443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.906421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a07b-cb11-4db8-b97c-d1f9a626443f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.906697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a07b-cb11-4db8-b97c-d1f9a626443f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.906965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a07b-cb11-4db8-b97c-d1f9a626443f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.907677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4a07b-cb11-4db8-b97c-d1f9a626443f None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=50ba627cb1e140bdafd9e390ff06ba3c, user.id=0b18297eab844ec19c7b4f770888e044) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:45.907980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4a07b-cb11-4db8-b97c-d1f9a626443f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.909420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4a07b-cb11-4db8-b97c-d1f9a626443f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.914944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 562/1106] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/role_assignments?user.id=0b18297eab844ec19c7b4f770888e044&scope.project.id=50ba627cb1e140bdafd9e390ff06ba3c&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:47.828890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25079d9-c8c9-40a1-b851-3a72c7b7abd8 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:47.829459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25079d9-c8c9-40a1-b851-3a72c7b7abd8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:47.829459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25079d9-c8c9-40a1-b851-3a72c7b7abd8 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:47.830789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 545/1107] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:47.836679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476decf3-ab7f-4101-9336-6ce95a414f56 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:47.837108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476decf3-ab7f-4101-9336-6ce95a414f56 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:47.837269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476decf3-ab7f-4101-9336-6ce95a414f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:47.855481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-476decf3-ab7f-4101-9336-6ce95a414f56 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:47.856647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-476decf3-ab7f-4101-9336-6ce95a414f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:47.878442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:47.922030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 563/1108] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:48.195743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25079d9-c8c9-40a1-b851-3a72c7b7abd8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.276412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:48.294618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25079d9-c8c9-40a1-b851-3a72c7b7abd8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.327930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd380a36-2dfc-46f0-b524-69407c76c421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.328544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd380a36-2dfc-46f0-b524-69407c76c421 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.328820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd380a36-2dfc-46f0-b524-69407c76c421 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.329097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd380a36-2dfc-46f0-b524-69407c76c421 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.330219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd380a36-2dfc-46f0-b524-69407c76c421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.331647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd380a36-2dfc-46f0-b524-69407c76c421 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.333239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 546/1109] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:48.338937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476decf3-ab7f-4101-9336-6ce95a414f56 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.431637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476decf3-ab7f-4101-9336-6ce95a414f56 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.464587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826230e3-8c83-45b1-a5a6-8f16f04c3538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.465135 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826230e3-8c83-45b1-a5a6-8f16f04c3538 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.465411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826230e3-8c83-45b1-a5a6-8f16f04c3538 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.465676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826230e3-8c83-45b1-a5a6-8f16f04c3538 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.466318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826230e3-8c83-45b1-a5a6-8f16f04c3538 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.467909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826230e3-8c83-45b1-a5a6-8f16f04c3538 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.487746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 564/1110] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:49:48 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:49:50.407752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a56667-f6e6-40d0-ad08-fc41f015066a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.408295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a56667-f6e6-40d0-ad08-fc41f015066a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.408295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a56667-f6e6-40d0-ad08-fc41f015066a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.409707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 547/1111] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:50.414729 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065361fb-fd26-4ffc-94b2-d5e752ec882f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.415139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065361fb-fd26-4ffc-94b2-d5e752ec882f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.415275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065361fb-fd26-4ffc-94b2-d5e752ec882f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.433486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-065361fb-fd26-4ffc-94b2-d5e752ec882f None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:50.434662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-065361fb-fd26-4ffc-94b2-d5e752ec882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:50.443801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:50.499831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 565/1112] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:50.770896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a56667-f6e6-40d0-ad08-fc41f015066a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.869515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a56667-f6e6-40d0-ad08-fc41f015066a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.902210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e5a61-d6b4-41e8-8a71-93de98cccf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.902818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e5a61-d6b4-41e8-8a71-93de98cccf91 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.903095 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e5a61-d6b4-41e8-8a71-93de98cccf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.903381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e5a61-d6b4-41e8-8a71-93de98cccf91 None admin] PATH_INFO: `/v3/roles/26e0da7be50c47de8620dca1c657eb4f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.904881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e5a61-d6b4-41e8-8a71-93de98cccf91 None admin] RBAC: Authorizing `identity:get_role(role_id=26e0da7be50c47de8620dca1c657eb4f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.906266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e5a61-d6b4-41e8-8a71-93de98cccf91 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.907567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 548/1113] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/roles/26e0da7be50c47de8620dca1c657eb4f => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:50.913343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065361fb-fd26-4ffc-94b2-d5e752ec882f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.009342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065361fb-fd26-4ffc-94b2-d5e752ec882f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.041174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80d848-3d7c-42ab-9754-5d142b3516b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.041722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80d848-3d7c-42ab-9754-5d142b3516b1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.041993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80d848-3d7c-42ab-9754-5d142b3516b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.042256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80d848-3d7c-42ab-9754-5d142b3516b1 None admin] PATH_INFO: `/v3/users/ceca806f12e5476dab5a9e9365478d44` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.055278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80d848-3d7c-42ab-9754-5d142b3516b1 None admin] RBAC: Authorizing `identity:get_user(user_id=ceca806f12e5476dab5a9e9365478d44)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.056697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80d848-3d7c-42ab-9754-5d142b3516b1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.058291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 566/1114] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/users/ceca806f12e5476dab5a9e9365478d44 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:51.064572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e5a61-d6b4-41e8-8a71-93de98cccf91 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.084322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.164210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e5a61-d6b4-41e8-8a71-93de98cccf91 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.197350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb97de-98a6-4398-88cd-802abfa46973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.197944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb97de-98a6-4398-88cd-802abfa46973 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.198225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb97de-98a6-4398-88cd-802abfa46973 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.198498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb97de-98a6-4398-88cd-802abfa46973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.263738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb97de-98a6-4398-88cd-802abfa46973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.265192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb97de-98a6-4398-88cd-802abfa46973 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.282573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.360084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 549/1115] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:51.365976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80d848-3d7c-42ab-9754-5d142b3516b1 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.368802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.461865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80d848-3d7c-42ab-9754-5d142b3516b1 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.494899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d680b7-37ec-43b6-b332-d4626e54ee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.495512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d680b7-37ec-43b6-b332-d4626e54ee7f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.495790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d680b7-37ec-43b6-b332-d4626e54ee7f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.496060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d680b7-37ec-43b6-b332-d4626e54ee7f None admin] PATH_INFO: `/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.497182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d680b7-37ec-43b6-b332-d4626e54ee7f None admin] RBAC: Authorizing `identity:get_project(project_id=50ba627cb1e140bdafd9e390ff06ba3c)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.498819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d680b7-37ec-43b6-b332-d4626e54ee7f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.500412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 567/1116] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:51.506660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb97de-98a6-4398-88cd-802abfa46973 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.604334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb97de-98a6-4398-88cd-802abfa46973 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.620901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.637937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3314c6-912d-4f68-9342-c917d0e6e899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.639106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3314c6-912d-4f68-9342-c917d0e6e899 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.639458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3314c6-912d-4f68-9342-c917d0e6e899 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.639728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3314c6-912d-4f68-9342-c917d0e6e899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.640442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3314c6-912d-4f68-9342-c917d0e6e899 None admin] RBAC: Adding query filter params (role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=50ba627cb1e140bdafd9e390ff06ba3c, user.id=ceca806f12e5476dab5a9e9365478d44) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:51.640746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3314c6-912d-4f68-9342-c917d0e6e899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.642186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3314c6-912d-4f68-9342-c917d0e6e899 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.647349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 550/1117] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/role_assignments?user.id=ceca806f12e5476dab5a9e9365478d44&scope.project.id=50ba627cb1e140bdafd9e390ff06ba3c&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:53.604861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06615e78-1795-4917-803c-7fc6ca6982ef None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.605400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06615e78-1795-4917-803c-7fc6ca6982ef None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.605400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06615e78-1795-4917-803c-7fc6ca6982ef None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.606704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 568/1118] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:53.611903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c045d-cade-4cd2-95c7-dc156c18423d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.612287 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c045d-cade-4cd2-95c7-dc156c18423d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.612443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c045d-cade-4cd2-95c7-dc156c18423d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.630534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-495c045d-cade-4cd2-95c7-dc156c18423d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:53.631696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-495c045d-cade-4cd2-95c7-dc156c18423d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:53.696950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 551/1119] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:53.963806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06615e78-1795-4917-803c-7fc6ca6982ef None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.030823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:54.061544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06615e78-1795-4917-803c-7fc6ca6982ef None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.093852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07f488-db21-4180-bdde-05ac3b1efeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.094458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07f488-db21-4180-bdde-05ac3b1efeda None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.094730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07f488-db21-4180-bdde-05ac3b1efeda None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.094995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07f488-db21-4180-bdde-05ac3b1efeda None admin] PATH_INFO: `/v3/roles/26e0da7be50c47de8620dca1c657eb4f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.096345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07f488-db21-4180-bdde-05ac3b1efeda None admin] RBAC: Authorizing `identity:get_role(role_id=26e0da7be50c47de8620dca1c657eb4f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.097720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07f488-db21-4180-bdde-05ac3b1efeda None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.099011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 569/1120] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:49:53 2022] GET /identity/v3/roles/26e0da7be50c47de8620dca1c657eb4f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:54.104801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c045d-cade-4cd2-95c7-dc156c18423d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.200774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c045d-cade-4cd2-95c7-dc156c18423d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.214045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:54.235770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa5eb4-4a78-4829-878a-56e276cb0f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.236384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa5eb4-4a78-4829-878a-56e276cb0f42 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.236662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa5eb4-4a78-4829-878a-56e276cb0f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.236932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa5eb4-4a78-4829-878a-56e276cb0f42 None admin] PATH_INFO: `/v3/users/ceca806f12e5476dab5a9e9365478d44` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.238076 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa5eb4-4a78-4829-878a-56e276cb0f42 None admin] RBAC: Authorizing `identity:get_user(user_id=ceca806f12e5476dab5a9e9365478d44)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.239535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa5eb4-4a78-4829-878a-56e276cb0f42 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.241140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 552/1121] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:49:54 2022] GET /identity/v3/users/ceca806f12e5476dab5a9e9365478d44 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:54.246873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07f488-db21-4180-bdde-05ac3b1efeda None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.345599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07f488-db21-4180-bdde-05ac3b1efeda None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.378668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e043fd-8501-4923-a83c-a869ac7d922d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.379285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e043fd-8501-4923-a83c-a869ac7d922d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.379563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e043fd-8501-4923-a83c-a869ac7d922d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.379829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e043fd-8501-4923-a83c-a869ac7d922d None admin] PATH_INFO: `/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.380928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e043fd-8501-4923-a83c-a869ac7d922d None admin] RBAC: Authorizing `identity:get_project(project_id=50ba627cb1e140bdafd9e390ff06ba3c)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.382376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e043fd-8501-4923-a83c-a869ac7d922d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.383989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 570/1122] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:54 2022] GET /identity/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:54.389983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa5eb4-4a78-4829-878a-56e276cb0f42 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.486836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa5eb4-4a78-4829-878a-56e276cb0f42 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.519298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b868c6-ae25-49d9-8867-1391e639223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.519937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b868c6-ae25-49d9-8867-1391e639223a None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.520216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b868c6-ae25-49d9-8867-1391e639223a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.520484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b868c6-ae25-49d9-8867-1391e639223a None admin] PATH_INFO: `/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c/users/ceca806f12e5476dab5a9e9365478d44/roles/26e0da7be50c47de8620dca1c657eb4f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.522692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b868c6-ae25-49d9-8867-1391e639223a None admin] RBAC: Authorizing `identity:create_grant(project_id=50ba627cb1e140bdafd9e390ff06ba3c, user_id=ceca806f12e5476dab5a9e9365478d44, role_id=26e0da7be50c47de8620dca1c657eb4f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.524099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b868c6-ae25-49d9-8867-1391e639223a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.536890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 553/1123] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:54 2022] PUT /identity/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c/users/ceca806f12e5476dab5a9e9365478d44/roles/26e0da7be50c47de8620dca1c657eb4f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:56.414439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a27cd-b1bf-46b8-983d-455787aa391c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.415018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a27cd-b1bf-46b8-983d-455787aa391c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.415018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a27cd-b1bf-46b8-983d-455787aa391c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.416311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 571/1124] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:56.422017 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8cd0c-7450-4a3e-b519-6b171fa7f5e5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.422448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8cd0c-7450-4a3e-b519-6b171fa7f5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.422549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8cd0c-7450-4a3e-b519-6b171fa7f5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.440563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-04b8cd0c-7450-4a3e-b519-6b171fa7f5e5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:56.441776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-04b8cd0c-7450-4a3e-b519-6b171fa7f5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:56.474687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.506254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 554/1125] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:56.778299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a27cd-b1bf-46b8-983d-455787aa391c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:56.787813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.879228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a27cd-b1bf-46b8-983d-455787aa391c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:56.913154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90f81-0d11-49cb-b402-562f06eeb348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:56.913762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90f81-0d11-49cb-b402-562f06eeb348 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.914034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90f81-0d11-49cb-b402-562f06eeb348 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.914297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90f81-0d11-49cb-b402-562f06eeb348 None admin] PATH_INFO: `/v3/roles/26e0da7be50c47de8620dca1c657eb4f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.915650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f90f81-0d11-49cb-b402-562f06eeb348 None admin] RBAC: Authorizing `identity:get_role(role_id=26e0da7be50c47de8620dca1c657eb4f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:56.916994 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f90f81-0d11-49cb-b402-562f06eeb348 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:56.918346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 572/1126] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/roles/26e0da7be50c47de8620dca1c657eb4f => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:56.924812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8cd0c-7450-4a3e-b519-6b171fa7f5e5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.024313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8cd0c-7450-4a3e-b519-6b171fa7f5e5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.056879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f71551-5b13-423d-836a-afeee74eafb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.057450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f71551-5b13-423d-836a-afeee74eafb1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.057724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f71551-5b13-423d-836a-afeee74eafb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.057990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f71551-5b13-423d-836a-afeee74eafb1 None admin] PATH_INFO: `/v3/users/ceca806f12e5476dab5a9e9365478d44` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.059081 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f71551-5b13-423d-836a-afeee74eafb1 None admin] RBAC: Authorizing `identity:get_user(user_id=ceca806f12e5476dab5a9e9365478d44)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.060509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f71551-5b13-423d-836a-afeee74eafb1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.062089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 555/1127] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/users/ceca806f12e5476dab5a9e9365478d44 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:57.068207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90f81-0d11-49cb-b402-562f06eeb348 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.141287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.165591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90f81-0d11-49cb-b402-562f06eeb348 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.197697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8ac88-94a9-4a09-8a24-4514fde75e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.198279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8ac88-94a9-4a09-8a24-4514fde75e66 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.198552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8ac88-94a9-4a09-8a24-4514fde75e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.198816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8ac88-94a9-4a09-8a24-4514fde75e66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.263173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8ac88-94a9-4a09-8a24-4514fde75e66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.264609 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8ac88-94a9-4a09-8a24-4514fde75e66 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.340703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.365630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 573/1128] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:57.371565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f71551-5b13-423d-836a-afeee74eafb1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.410599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.468915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f71551-5b13-423d-836a-afeee74eafb1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.501999 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68675919-2ff3-4ce2-932d-186911ebbcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.502608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68675919-2ff3-4ce2-932d-186911ebbcbd None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.502887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68675919-2ff3-4ce2-932d-186911ebbcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.503158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68675919-2ff3-4ce2-932d-186911ebbcbd None admin] PATH_INFO: `/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.504268 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68675919-2ff3-4ce2-932d-186911ebbcbd None admin] RBAC: Authorizing `identity:get_project(project_id=50ba627cb1e140bdafd9e390ff06ba3c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.505713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68675919-2ff3-4ce2-932d-186911ebbcbd None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.507345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 556/1129] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/projects/50ba627cb1e140bdafd9e390ff06ba3c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:57.513538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8ac88-94a9-4a09-8a24-4514fde75e66 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.609861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8ac88-94a9-4a09-8a24-4514fde75e66 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.642482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4a5e9-aae5-4fb5-8ea5-bf7ad8576313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.643069 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4a5e9-aae5-4fb5-8ea5-bf7ad8576313 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.643345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4a5e9-aae5-4fb5-8ea5-bf7ad8576313 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.643612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4a5e9-aae5-4fb5-8ea5-bf7ad8576313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.644319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4a5e9-aae5-4fb5-8ea5-bf7ad8576313 None admin] RBAC: Adding query filter params (role.id=26e0da7be50c47de8620dca1c657eb4f, scope.project.id=50ba627cb1e140bdafd9e390ff06ba3c, user.id=ceca806f12e5476dab5a9e9365478d44) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:57.644622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4a5e9-aae5-4fb5-8ea5-bf7ad8576313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.646058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4a5e9-aae5-4fb5-8ea5-bf7ad8576313 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.651565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 574/1130] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/role_assignments?user.id=ceca806f12e5476dab5a9e9365478d44&scope.project.id=50ba627cb1e140bdafd9e390ff06ba3c&role.id=26e0da7be50c47de8620dca1c657eb4f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:59.563005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826f8e5-ee27-4a30-b42d-789ee51748c4 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:59.563533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826f8e5-ee27-4a30-b42d-789ee51748c4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:59.563533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826f8e5-ee27-4a30-b42d-789ee51748c4 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:59.564792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 557/1131] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:59.570152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44939c56-4343-435e-8c12-405d140289c3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:59.570533 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44939c56-4343-435e-8c12-405d140289c3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:59.570699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44939c56-4343-435e-8c12-405d140289c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:59.588942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-44939c56-4343-435e-8c12-405d140289c3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:59.590155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-44939c56-4343-435e-8c12-405d140289c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:59.618651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:59.655144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 575/1132] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:59.936614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826f8e5-ee27-4a30-b42d-789ee51748c4 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:00.035729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826f8e5-ee27-4a30-b42d-789ee51748c4 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:00.042696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:00.068936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25ba56-99c1-45a1-ad44-4cf3eb37c19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:00.069575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25ba56-99c1-45a1-ad44-4cf3eb37c19c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.069854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25ba56-99c1-45a1-ad44-4cf3eb37c19c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.070122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25ba56-99c1-45a1-ad44-4cf3eb37c19c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.071222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25ba56-99c1-45a1-ad44-4cf3eb37c19c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:00.072631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25ba56-99c1-45a1-ad44-4cf3eb37c19c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:00.074233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 558/1133] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:00.080983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44939c56-4343-435e-8c12-405d140289c3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:00.178329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44939c56-4343-435e-8c12-405d140289c3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:00.212258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736fd5d-e3bb-44b4-bbc2-6b21cc5f233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:00.212793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736fd5d-e3bb-44b4-bbc2-6b21cc5f233e None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.213090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736fd5d-e3bb-44b4-bbc2-6b21cc5f233e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.213369 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736fd5d-e3bb-44b4-bbc2-6b21cc5f233e None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.214016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736fd5d-e3bb-44b4-bbc2-6b21cc5f233e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:00.215437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736fd5d-e3bb-44b4-bbc2-6b21cc5f233e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:00.241608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 576/1134] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:50:00 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:02.159110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d512930-d052-430d-879c-5cd60224cf36 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.159669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d512930-d052-430d-879c-5cd60224cf36 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.159669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d512930-d052-430d-879c-5cd60224cf36 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.160991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 559/1135] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:02.165988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99743582-7c7a-4a33-b9ee-67abafb4fe82 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.166413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99743582-7c7a-4a33-b9ee-67abafb4fe82 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.166518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99743582-7c7a-4a33-b9ee-67abafb4fe82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.186050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-99743582-7c7a-4a33-b9ee-67abafb4fe82 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:02.187222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-99743582-7c7a-4a33-b9ee-67abafb4fe82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:02.200970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:02.252680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 577/1136] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:02.525591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d512930-d052-430d-879c-5cd60224cf36 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.624097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d512930-d052-430d-879c-5cd60224cf36 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:02.657375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfa797-918a-4dfa-8f24-1a7b6c7ffa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:02.657974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfa797-918a-4dfa-8f24-1a7b6c7ffa71 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.658248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfa797-918a-4dfa-8f24-1a7b6c7ffa71 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.658514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfa797-918a-4dfa-8f24-1a7b6c7ffa71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.659595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfa797-918a-4dfa-8f24-1a7b6c7ffa71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:02.660998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfa797-918a-4dfa-8f24-1a7b6c7ffa71 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:02.662560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 560/1137] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:02.668285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99743582-7c7a-4a33-b9ee-67abafb4fe82 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.767299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99743582-7c7a-4a33-b9ee-67abafb4fe82 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:02.801863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae117cb-ac75-4825-9a74-bebce385eeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:02.802426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae117cb-ac75-4825-9a74-bebce385eeb5 None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.802704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae117cb-ac75-4825-9a74-bebce385eeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.802971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae117cb-ac75-4825-9a74-bebce385eeb5 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.803616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae117cb-ac75-4825-9a74-bebce385eeb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:02.805033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae117cb-ac75-4825-9a74-bebce385eeb5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:02.827151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 578/1138] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:50:02 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:04.788103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bf8f0-0e25-4853-a2fd-3d4e10126a72 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:04.788671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bf8f0-0e25-4853-a2fd-3d4e10126a72 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:04.788671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bf8f0-0e25-4853-a2fd-3d4e10126a72 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:04.789998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 561/1139] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:04.795315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cada02-0750-4c9c-b419-042553738008 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:04.795743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cada02-0750-4c9c-b419-042553738008 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:04.796176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cada02-0750-4c9c-b419-042553738008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:04.809734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:04.815384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-68cada02-0750-4c9c-b419-042553738008 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:04.817133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-68cada02-0750-4c9c-b419-042553738008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:04.881027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 579/1140] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:05.150558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bf8f0-0e25-4853-a2fd-3d4e10126a72 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:05.195298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:05.251885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bf8f0-0e25-4853-a2fd-3d4e10126a72 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:05.285886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ab7f0-7b1d-45bd-8b8a-2d6302127424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:05.286506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ab7f0-7b1d-45bd-8b8a-2d6302127424 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.286810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ab7f0-7b1d-45bd-8b8a-2d6302127424 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.287079 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ab7f0-7b1d-45bd-8b8a-2d6302127424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.296936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ab7f0-7b1d-45bd-8b8a-2d6302127424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-970ab7f0-7b1d-45bd-8b8a-2d6302127424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:05.301809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:05.302929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:50:05.303884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 562/1141] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:05.309529 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cada02-0750-4c9c-b419-042553738008 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:05.405242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cada02-0750-4c9c-b419-042553738008 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:05.438436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33161f-ba11-41f8-972c-b2abde46e557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:05.439028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33161f-ba11-41f8-972c-b2abde46e557 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.439311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33161f-ba11-41f8-972c-b2abde46e557 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.439587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33161f-ba11-41f8-972c-b2abde46e557 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.440246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33161f-ba11-41f8-972c-b2abde46e557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:05.440546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33161f-ba11-41f8-972c-b2abde46e557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:05.441932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33161f-ba11-41f8-972c-b2abde46e557 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:05.444313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:05.451473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 580/1142] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:05.457418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ab7f0-7b1d-45bd-8b8a-2d6302127424 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:05.554786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ab7f0-7b1d-45bd-8b8a-2d6302127424 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:05.556277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:05.588564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e52465-b151-4613-803f-ad64fe0a2a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:05.589206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e52465-b151-4613-803f-ad64fe0a2a42 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.589485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e52465-b151-4613-803f-ad64fe0a2a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.589753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e52465-b151-4613-803f-ad64fe0a2a42 None admin] PATH_INFO: `/v3/users/d3309ffd5fec474fa2cdd8e34072a9c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.602756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e52465-b151-4613-803f-ad64fe0a2a42 None admin] RBAC: Authorizing `identity:get_user(user_id=d3309ffd5fec474fa2cdd8e34072a9c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:05.604170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e52465-b151-4613-803f-ad64fe0a2a42 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:05.605788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 563/1143] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:05 2022] GET /identity/v3/users/d3309ffd5fec474fa2cdd8e34072a9c4 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:05.612382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33161f-ba11-41f8-972c-b2abde46e557 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:05.709346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33161f-ba11-41f8-972c-b2abde46e557 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:05.743012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430e638-f2eb-431e-9220-b65527a1c537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:05.743608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430e638-f2eb-431e-9220-b65527a1c537 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.743888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430e638-f2eb-431e-9220-b65527a1c537 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.744160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430e638-f2eb-431e-9220-b65527a1c537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.812154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:05.816324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430e638-f2eb-431e-9220-b65527a1c537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:05.818439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430e638-f2eb-431e-9220-b65527a1c537 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:05.921355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 581/1144] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:05.927270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e52465-b151-4613-803f-ad64fe0a2a42 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.039184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e52465-b151-4613-803f-ad64fe0a2a42 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.071610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37f81e-e8c1-4ea4-86ec-84ffebc776dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.072227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37f81e-e8c1-4ea4-86ec-84ffebc776dc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.072508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37f81e-e8c1-4ea4-86ec-84ffebc776dc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.072782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37f81e-e8c1-4ea4-86ec-84ffebc776dc None admin] PATH_INFO: `/v3/projects/be56fd49dfec45068e71d9c631648057` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.073921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37f81e-e8c1-4ea4-86ec-84ffebc776dc None admin] RBAC: Authorizing `identity:get_project(project_id=be56fd49dfec45068e71d9c631648057)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.075866 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37f81e-e8c1-4ea4-86ec-84ffebc776dc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.077541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 564/1145] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:05 2022] GET /identity/v3/projects/be56fd49dfec45068e71d9c631648057 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:06.083731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430e638-f2eb-431e-9220-b65527a1c537 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.176392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:06.180085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430e638-f2eb-431e-9220-b65527a1c537 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.214837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f1c44-e144-4e5d-a0e7-f8febd8917fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.215434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f1c44-e144-4e5d-a0e7-f8febd8917fa None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.215754 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f1c44-e144-4e5d-a0e7-f8febd8917fa None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.216009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f1c44-e144-4e5d-a0e7-f8febd8917fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.216719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f1c44-e144-4e5d-a0e7-f8febd8917fa None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=be56fd49dfec45068e71d9c631648057, user.id=d3309ffd5fec474fa2cdd8e34072a9c4) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:06.217023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f1c44-e144-4e5d-a0e7-f8febd8917fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.218469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f1c44-e144-4e5d-a0e7-f8febd8917fa None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.223689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 582/1146] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/role_assignments?user.id=d3309ffd5fec474fa2cdd8e34072a9c4&scope.project.id=be56fd49dfec45068e71d9c631648057&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:08.113333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8568f9-72ec-4508-b5c0-0b94f8564154 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:08.113895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8568f9-72ec-4508-b5c0-0b94f8564154 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:08.113895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8568f9-72ec-4508-b5c0-0b94f8564154 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:08.115199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 565/1147] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:08.120361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f13f8-0190-4cb7-9963-46ce1b8b66e9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:08.120759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f13f8-0190-4cb7-9963-46ce1b8b66e9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:08.120911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f13f8-0190-4cb7-9963-46ce1b8b66e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:08.139324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0b9f13f8-0190-4cb7-9963-46ce1b8b66e9 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:08.140494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0b9f13f8-0190-4cb7-9963-46ce1b8b66e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:08.205673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 583/1148] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:08.472923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8568f9-72ec-4508-b5c0-0b94f8564154 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:08.500955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:08.571409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8568f9-72ec-4508-b5c0-0b94f8564154 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:08.603520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66f606-17a4-457e-8acb-21c801874551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:08.604123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66f606-17a4-457e-8acb-21c801874551 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:08.604395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66f606-17a4-457e-8acb-21c801874551 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:08.604660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66f606-17a4-457e-8acb-21c801874551 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:08.615057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66f606-17a4-457e-8acb-21c801874551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9c66f606-17a4-457e-8acb-21c801874551 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:08.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:08.620803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:50:08.621875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 566/1149] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:08.626732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f13f8-0190-4cb7-9963-46ce1b8b66e9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:08.712978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:08.722273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f13f8-0190-4cb7-9963-46ce1b8b66e9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:08.754959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb3b1d-eac0-41a4-8c89-79e8bd8bcce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:08.755494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb3b1d-eac0-41a4-8c89-79e8bd8bcce7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:08.755770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb3b1d-eac0-41a4-8c89-79e8bd8bcce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:08.756038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb3b1d-eac0-41a4-8c89-79e8bd8bcce7 None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:08.756693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb3b1d-eac0-41a4-8c89-79e8bd8bcce7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:08.756991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb3b1d-eac0-41a4-8c89-79e8bd8bcce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:08.758372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb3b1d-eac0-41a4-8c89-79e8bd8bcce7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:08.766686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 584/1150] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:08.772431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66f606-17a4-457e-8acb-21c801874551 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:08.860851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:08.873277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66f606-17a4-457e-8acb-21c801874551 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:08.906902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42a94d-878b-418f-a1d3-bec92d7b1192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:08.907517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a94d-878b-418f-a1d3-bec92d7b1192 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:08.907796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a94d-878b-418f-a1d3-bec92d7b1192 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:08.908070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a94d-878b-418f-a1d3-bec92d7b1192 None admin] PATH_INFO: `/v3/users/d3309ffd5fec474fa2cdd8e34072a9c4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:08.909212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42a94d-878b-418f-a1d3-bec92d7b1192 None admin] RBAC: Authorizing `identity:get_user(user_id=d3309ffd5fec474fa2cdd8e34072a9c4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:08.910636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42a94d-878b-418f-a1d3-bec92d7b1192 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:08.912212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 567/1151] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:08 2022] GET /identity/v3/users/d3309ffd5fec474fa2cdd8e34072a9c4 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:08.918017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb3b1d-eac0-41a4-8c89-79e8bd8bcce7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:09.018207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb3b1d-eac0-41a4-8c89-79e8bd8bcce7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:09.050661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da175293-740c-48d7-a8b2-ca59ec8980f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:09.051271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da175293-740c-48d7-a8b2-ca59ec8980f1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:09.051546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da175293-740c-48d7-a8b2-ca59ec8980f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:09.051819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da175293-740c-48d7-a8b2-ca59ec8980f1 None admin] PATH_INFO: `/v3/projects/be56fd49dfec45068e71d9c631648057` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:09.052911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da175293-740c-48d7-a8b2-ca59ec8980f1 None admin] RBAC: Authorizing `identity:get_project(project_id=be56fd49dfec45068e71d9c631648057)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:09.054366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da175293-740c-48d7-a8b2-ca59ec8980f1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:09.055968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 585/1152] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:08 2022] GET /identity/v3/projects/be56fd49dfec45068e71d9c631648057 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:09.062039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42a94d-878b-418f-a1d3-bec92d7b1192 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:09.159889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42a94d-878b-418f-a1d3-bec92d7b1192 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:09.193159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6faab-ead6-4629-a01a-591dd38eb422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:09.193793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6faab-ead6-4629-a01a-591dd38eb422 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:09.194066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6faab-ead6-4629-a01a-591dd38eb422 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:09.194333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6faab-ead6-4629-a01a-591dd38eb422 None admin] PATH_INFO: `/v3/projects/be56fd49dfec45068e71d9c631648057/users/d3309ffd5fec474fa2cdd8e34072a9c4/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:09.196203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6faab-ead6-4629-a01a-591dd38eb422 None admin] RBAC: Authorizing `identity:create_grant(project_id=be56fd49dfec45068e71d9c631648057, user_id=d3309ffd5fec474fa2cdd8e34072a9c4, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:09.197637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6faab-ead6-4629-a01a-591dd38eb422 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:09.211790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 568/1153] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:50:09 2022] PUT /identity/v3/projects/be56fd49dfec45068e71d9c631648057/users/d3309ffd5fec474fa2cdd8e34072a9c4/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:50:11.091439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a53d21-dd99-4c5a-9952-b98dde4ee765 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.091976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a53d21-dd99-4c5a-9952-b98dde4ee765 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.091976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a53d21-dd99-4c5a-9952-b98dde4ee765 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.093479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 586/1154] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:11.098592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fc01b-d615-4508-82cd-b07aea2e103b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.098994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fc01b-d615-4508-82cd-b07aea2e103b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.099131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fc01b-d615-4508-82cd-b07aea2e103b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.117365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a34fc01b-d615-4508-82cd-b07aea2e103b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:11.118559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a34fc01b-d615-4508-82cd-b07aea2e103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:11.129402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.183151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 569/1155] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:11.448053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a53d21-dd99-4c5a-9952-b98dde4ee765 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.472864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.549727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a53d21-dd99-4c5a-9952-b98dde4ee765 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.581982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9a8b6-b798-4163-b374-4e4fc24566c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.582586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9a8b6-b798-4163-b374-4e4fc24566c8 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.582859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9a8b6-b798-4163-b374-4e4fc24566c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.583126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9a8b6-b798-4163-b374-4e4fc24566c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.593390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9a8b6-b798-4163-b374-4e4fc24566c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3ab9a8b6-b798-4163-b374-4e4fc24566c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:11.598044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:11.599105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:50:11.599958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 587/1156] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:11.605147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fc01b-d615-4508-82cd-b07aea2e103b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.702868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fc01b-d615-4508-82cd-b07aea2e103b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.736450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5f949-afb9-41ab-870f-8b375ecda206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.737032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5f949-afb9-41ab-870f-8b375ecda206 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.737340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5f949-afb9-41ab-870f-8b375ecda206 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.737610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5f949-afb9-41ab-870f-8b375ecda206 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.738263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5f949-afb9-41ab-870f-8b375ecda206 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:11.738562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5f949-afb9-41ab-870f-8b375ecda206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:11.739904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5f949-afb9-41ab-870f-8b375ecda206 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:11.747970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 570/1157] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:11.753749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9a8b6-b798-4163-b374-4e4fc24566c8 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.834192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.852259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9a8b6-b798-4163-b374-4e4fc24566c8 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.886234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ef334-582b-4423-a73d-bdeaf10b7f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.886850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ef334-582b-4423-a73d-bdeaf10b7f51 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.887128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ef334-582b-4423-a73d-bdeaf10b7f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.888033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ef334-582b-4423-a73d-bdeaf10b7f51 None admin] PATH_INFO: `/v3/users/d3309ffd5fec474fa2cdd8e34072a9c4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.889326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ef334-582b-4423-a73d-bdeaf10b7f51 None admin] RBAC: Authorizing `identity:get_user(user_id=d3309ffd5fec474fa2cdd8e34072a9c4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:11.890749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ef334-582b-4423-a73d-bdeaf10b7f51 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:11.892323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 588/1158] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/users/d3309ffd5fec474fa2cdd8e34072a9c4 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:11.898441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5f949-afb9-41ab-870f-8b375ecda206 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.910675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.996771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5f949-afb9-41ab-870f-8b375ecda206 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:12.033727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2bd93-e239-4fa7-91ea-aead07fc1006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:12.034323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2bd93-e239-4fa7-91ea-aead07fc1006 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:12.034609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2bd93-e239-4fa7-91ea-aead07fc1006 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:12.034873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2bd93-e239-4fa7-91ea-aead07fc1006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:12.099848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2bd93-e239-4fa7-91ea-aead07fc1006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:12.101307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2bd93-e239-4fa7-91ea-aead07fc1006 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:12.110567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:12.196970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 571/1159] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:12.204012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ef334-582b-4423-a73d-bdeaf10b7f51 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:12.304882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ef334-582b-4423-a73d-bdeaf10b7f51 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:12.338377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103694a-2882-4593-8325-c0a02f550c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:12.338990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103694a-2882-4593-8325-c0a02f550c5b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:12.339266 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103694a-2882-4593-8325-c0a02f550c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:12.339532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103694a-2882-4593-8325-c0a02f550c5b None admin] PATH_INFO: `/v3/projects/be56fd49dfec45068e71d9c631648057` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:12.341443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103694a-2882-4593-8325-c0a02f550c5b None admin] RBAC: Authorizing `identity:get_project(project_id=be56fd49dfec45068e71d9c631648057)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:12.342928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103694a-2882-4593-8325-c0a02f550c5b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:12.344538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 589/1160] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:12 2022] GET /identity/v3/projects/be56fd49dfec45068e71d9c631648057 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:12.350738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2bd93-e239-4fa7-91ea-aead07fc1006 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:12.446311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2bd93-e239-4fa7-91ea-aead07fc1006 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:12.455245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:12.479768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba2018-6b1e-4438-88c2-674c93ba3ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:12.480347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba2018-6b1e-4438-88c2-674c93ba3ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:12.480622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba2018-6b1e-4438-88c2-674c93ba3ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:12.480885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba2018-6b1e-4438-88c2-674c93ba3ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:12.482300 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba2018-6b1e-4438-88c2-674c93ba3ef0 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=be56fd49dfec45068e71d9c631648057, user.id=d3309ffd5fec474fa2cdd8e34072a9c4) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:12.482601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba2018-6b1e-4438-88c2-674c93ba3ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:12.484002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba2018-6b1e-4438-88c2-674c93ba3ef0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:12.489553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 572/1161] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:12 2022] GET /identity/v3/role_assignments?user.id=d3309ffd5fec474fa2cdd8e34072a9c4&scope.project.id=be56fd49dfec45068e71d9c631648057&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:14.429049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4594f-3fdf-4d42-8fae-6d7b8e88f208 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:14.429687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4594f-3fdf-4d42-8fae-6d7b8e88f208 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:14.429687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4594f-3fdf-4d42-8fae-6d7b8e88f208 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:14.430962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 590/1162] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:14.436731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3481efa-a5b0-4a4a-9021-bda4cf7f59e5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:14.437172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3481efa-a5b0-4a4a-9021-bda4cf7f59e5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:14.437295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3481efa-a5b0-4a4a-9021-bda4cf7f59e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:14.455818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c3481efa-a5b0-4a4a-9021-bda4cf7f59e5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:14.456983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c3481efa-a5b0-4a4a-9021-bda4cf7f59e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:14.519617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 573/1163] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:14.792382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4594f-3fdf-4d42-8fae-6d7b8e88f208 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:14.812491 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:14.893598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4594f-3fdf-4d42-8fae-6d7b8e88f208 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:14.927237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337701e-7c92-45cc-8c93-46ad6777a5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:14.927833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337701e-7c92-45cc-8c93-46ad6777a5ad None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:14.928110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337701e-7c92-45cc-8c93-46ad6777a5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:14.928379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337701e-7c92-45cc-8c93-46ad6777a5ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:14.993978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337701e-7c92-45cc-8c93-46ad6777a5ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:14.995411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337701e-7c92-45cc-8c93-46ad6777a5ad None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:15.012993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:15.093143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 591/1164] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:15.099014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3481efa-a5b0-4a4a-9021-bda4cf7f59e5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:15.196931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3481efa-a5b0-4a4a-9021-bda4cf7f59e5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:15.198413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:15.231004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc52fa7-af22-40bd-b634-20dd2b065723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:15.231614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc52fa7-af22-40bd-b634-20dd2b065723 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:15.231885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc52fa7-af22-40bd-b634-20dd2b065723 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:15.232150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc52fa7-af22-40bd-b634-20dd2b065723 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:15.238624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc52fa7-af22-40bd-b634-20dd2b065723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:15.240051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc52fa7-af22-40bd-b634-20dd2b065723 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-8bc52fa7-af22-40bd-b634-20dd2b065723 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:15.251867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:15.253206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:50:15.254074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 574/1165] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:50:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:15.259092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337701e-7c92-45cc-8c93-46ad6777a5ad None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:15.357297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337701e-7c92-45cc-8c93-46ad6777a5ad None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:15.375472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:15.392342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134f350-6677-4bcf-bb04-ad6e5e8bf3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:15.392939 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134f350-6677-4bcf-bb04-ad6e5e8bf3fa None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:15.393247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134f350-6677-4bcf-bb04-ad6e5e8bf3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:15.393507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134f350-6677-4bcf-bb04-ad6e5e8bf3fa None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:15.394136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134f350-6677-4bcf-bb04-ad6e5e8bf3fa None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:15.394430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134f350-6677-4bcf-bb04-ad6e5e8bf3fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:15.395773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134f350-6677-4bcf-bb04-ad6e5e8bf3fa None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:15.402673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 592/1166] 10.222.0.78 () {56 vars in 1115 bytes} [Tue Aug 30 00:50:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:17.278367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0dfad-19f4-4a0f-96f5-3eb2cd3de0a9 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.278933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0dfad-19f4-4a0f-96f5-3eb2cd3de0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.278933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0dfad-19f4-4a0f-96f5-3eb2cd3de0a9 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.280224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 575/1167] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:17.285285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a302-a7c7-46d1-9bf5-f882005fa89e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.285696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a302-a7c7-46d1-9bf5-f882005fa89e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.285832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a302-a7c7-46d1-9bf5-f882005fa89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.303676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7169a302-a7c7-46d1-9bf5-f882005fa89e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:17.304862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7169a302-a7c7-46d1-9bf5-f882005fa89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:17.370779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 593/1168] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:17.643531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0dfad-19f4-4a0f-96f5-3eb2cd3de0a9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:17.742779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0dfad-19f4-4a0f-96f5-3eb2cd3de0a9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:17.775799 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d41ab-e357-4c40-9540-460d646f79ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:17.776390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d41ab-e357-4c40-9540-460d646f79ed None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.776672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d41ab-e357-4c40-9540-460d646f79ed None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.776944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d41ab-e357-4c40-9540-460d646f79ed None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.777723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d41ab-e357-4c40-9540-460d646f79ed None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:17.779213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d41ab-e357-4c40-9540-460d646f79ed None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:17.795677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 576/1169] 10.222.0.78 () {60 vars in 1138 bytes} [Tue Aug 30 00:50:17 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:19.742343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613654a-1ec8-4454-a379-e180fee3e19b None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:19.742942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613654a-1ec8-4454-a379-e180fee3e19b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:19.742942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613654a-1ec8-4454-a379-e180fee3e19b None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:19.744241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 594/1170] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:19.750262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec4f26-0acd-442f-8b5a-8070376093eb None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:19.750701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec4f26-0acd-442f-8b5a-8070376093eb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:19.750806 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec4f26-0acd-442f-8b5a-8070376093eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:19.769616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-62ec4f26-0acd-442f-8b5a-8070376093eb None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:19.770788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-62ec4f26-0acd-442f-8b5a-8070376093eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:19.786864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:19.837861 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 577/1171] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:20.114079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613654a-1ec8-4454-a379-e180fee3e19b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:20.215774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613654a-1ec8-4454-a379-e180fee3e19b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:20.228379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:20.250713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb80ca-00a4-44e4-84d1-b9a0d11ca9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:20.251341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb80ca-00a4-44e4-84d1-b9a0d11ca9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:20.251619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb80ca-00a4-44e4-84d1-b9a0d11ca9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:20.251892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb80ca-00a4-44e4-84d1-b9a0d11ca9d1 None admin] PATH_INFO: `/v3/domains/829997d9d0f04952936f2bf17d40ba78` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:20.262460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb80ca-00a4-44e4-84d1-b9a0d11ca9d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=829997d9d0f04952936f2bf17d40ba78)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:20.263903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb80ca-00a4-44e4-84d1-b9a0d11ca9d1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:20.265564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 595/1172] 10.222.0.78 () {56 vars in 1183 bytes} [Tue Aug 30 00:50:20 2022] GET /identity/v3/domains/829997d9d0f04952936f2bf17d40ba78 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:20.271351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec4f26-0acd-442f-8b5a-8070376093eb None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:20.371783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec4f26-0acd-442f-8b5a-8070376093eb None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:20.405192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a93f5d-35d2-43c1-b229-b94951cc5213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:20.405794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a93f5d-35d2-43c1-b229-b94951cc5213 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:20.406073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a93f5d-35d2-43c1-b229-b94951cc5213 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:20.406343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a93f5d-35d2-43c1-b229-b94951cc5213 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:20.406993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a93f5d-35d2-43c1-b229-b94951cc5213 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:20.408419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a93f5d-35d2-43c1-b229-b94951cc5213 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:20.434498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 578/1173] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:50:20 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:22.343280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67878a-1a74-46c9-9a13-e6a51c06016a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.343811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67878a-1a74-46c9-9a13-e6a51c06016a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.343811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67878a-1a74-46c9-9a13-e6a51c06016a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.345194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 596/1174] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:22.350504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeda026-e340-4b47-8e79-31024323bf56 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.350924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeda026-e340-4b47-8e79-31024323bf56 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.351051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeda026-e340-4b47-8e79-31024323bf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.358723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:22.369939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8aeda026-e340-4b47-8e79-31024323bf56 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:22.371161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8aeda026-e340-4b47-8e79-31024323bf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:22.438079 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 579/1175] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:22.713292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67878a-1a74-46c9-9a13-e6a51c06016a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:22.814014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67878a-1a74-46c9-9a13-e6a51c06016a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:22.847428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ce0bd-7900-418f-9ee6-423e7fb6ce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:22.848037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ce0bd-7900-418f-9ee6-423e7fb6ce2d None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.848318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ce0bd-7900-418f-9ee6-423e7fb6ce2d None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.848591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ce0bd-7900-418f-9ee6-423e7fb6ce2d None admin] PATH_INFO: `/v3/domains/829997d9d0f04952936f2bf17d40ba78` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.849694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ce0bd-7900-418f-9ee6-423e7fb6ce2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=829997d9d0f04952936f2bf17d40ba78)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:22.851125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ce0bd-7900-418f-9ee6-423e7fb6ce2d None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:22.852665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 597/1176] 10.222.0.78 () {56 vars in 1183 bytes} [Tue Aug 30 00:50:22 2022] GET /identity/v3/domains/829997d9d0f04952936f2bf17d40ba78 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:22.859019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeda026-e340-4b47-8e79-31024323bf56 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:22.952448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeda026-e340-4b47-8e79-31024323bf56 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:22.985501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06ca18-082c-4777-9530-ed5810a7238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:22.986017 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06ca18-082c-4777-9530-ed5810a7238e None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.986288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06ca18-082c-4777-9530-ed5810a7238e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.986551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06ca18-082c-4777-9530-ed5810a7238e None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.987179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06ca18-082c-4777-9530-ed5810a7238e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:22.988574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06ca18-082c-4777-9530-ed5810a7238e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:22.996405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:23.010271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 580/1177] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:50:22 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:24.943846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9ac4c-6362-4eb8-aebc-1dae56c4a0bf None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:24.944410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9ac4c-6362-4eb8-aebc-1dae56c4a0bf None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:24.944410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9ac4c-6362-4eb8-aebc-1dae56c4a0bf None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:24.945818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 598/1178] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:24.951465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dfdfe-a87b-4554-900e-5dcb05c0e21e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:24.951888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dfdfe-a87b-4554-900e-5dcb05c0e21e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:24.951985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dfdfe-a87b-4554-900e-5dcb05c0e21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:24.969818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ec4dfdfe-a87b-4554-900e-5dcb05c0e21e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:24.970980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ec4dfdfe-a87b-4554-900e-5dcb05c0e21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:25.034980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 581/1179] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:25.305047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9ac4c-6362-4eb8-aebc-1dae56c4a0bf None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.348031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:25.406991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9ac4c-6362-4eb8-aebc-1dae56c4a0bf None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.440552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc0e6b-8216-4a4e-8264-822e953e0b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.441194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc0e6b-8216-4a4e-8264-822e953e0b7b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.441476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc0e6b-8216-4a4e-8264-822e953e0b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.441749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc0e6b-8216-4a4e-8264-822e953e0b7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.451990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc0e6b-8216-4a4e-8264-822e953e0b7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-88cc0e6b-8216-4a4e-8264-822e953e0b7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:25.456717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:25.457870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:50:25.459192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 599/1180] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:25.463845 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dfdfe-a87b-4554-900e-5dcb05c0e21e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.560498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dfdfe-a87b-4554-900e-5dcb05c0e21e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.579933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:25.594167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1b2d5-b538-4ff5-bff2-5b6ad90b4a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.594761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1b2d5-b538-4ff5-bff2-5b6ad90b4a1c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.595040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1b2d5-b538-4ff5-bff2-5b6ad90b4a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.595310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1b2d5-b538-4ff5-bff2-5b6ad90b4a1c None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.595967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1b2d5-b538-4ff5-bff2-5b6ad90b4a1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:25.596272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1b2d5-b538-4ff5-bff2-5b6ad90b4a1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.597669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1b2d5-b538-4ff5-bff2-5b6ad90b4a1c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.606045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 582/1181] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:25.612050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc0e6b-8216-4a4e-8264-822e953e0b7b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.707279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:25.710268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc0e6b-8216-4a4e-8264-822e953e0b7b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.744987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766803e-8bdd-47ea-bf7f-80e882d9f4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.745633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766803e-8bdd-47ea-bf7f-80e882d9f4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.745951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766803e-8bdd-47ea-bf7f-80e882d9f4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.746310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766803e-8bdd-47ea-bf7f-80e882d9f4e3 None admin] PATH_INFO: `/v3/users/c303dffe4dfb4a0887849abd8a69b125` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.758820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766803e-8bdd-47ea-bf7f-80e882d9f4e3 None admin] RBAC: Authorizing `identity:get_user(user_id=c303dffe4dfb4a0887849abd8a69b125)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.760254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766803e-8bdd-47ea-bf7f-80e882d9f4e3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.761952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 600/1182] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/users/c303dffe4dfb4a0887849abd8a69b125 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:25.767869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1b2d5-b538-4ff5-bff2-5b6ad90b4a1c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.864248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1b2d5-b538-4ff5-bff2-5b6ad90b4a1c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.895763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c6d94-da35-4c4c-874a-030123ebf43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.896343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6d94-da35-4c4c-874a-030123ebf43d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.896617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6d94-da35-4c4c-874a-030123ebf43d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.896882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6d94-da35-4c4c-874a-030123ebf43d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.945064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:25.963058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c6d94-da35-4c4c-874a-030123ebf43d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.964504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c6d94-da35-4c4c-874a-030123ebf43d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:26.061321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 583/1183] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:26.067482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766803e-8bdd-47ea-bf7f-80e882d9f4e3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:26.165206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766803e-8bdd-47ea-bf7f-80e882d9f4e3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:26.198988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4adb9-e32d-4aec-b31d-7ec3babc39ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:26.200214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4adb9-e32d-4aec-b31d-7ec3babc39ce None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:26.200495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4adb9-e32d-4aec-b31d-7ec3babc39ce None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:26.200768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4adb9-e32d-4aec-b31d-7ec3babc39ce None admin] PATH_INFO: `/v3/projects/005292fef03343c9a1dcb831e4d817e1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:26.201913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4adb9-e32d-4aec-b31d-7ec3babc39ce None admin] RBAC: Authorizing `identity:get_project(project_id=005292fef03343c9a1dcb831e4d817e1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:26.203342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4adb9-e32d-4aec-b31d-7ec3babc39ce None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:26.204987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 601/1184] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:26 2022] GET /identity/v3/projects/005292fef03343c9a1dcb831e4d817e1 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:26.211330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c6d94-da35-4c4c-874a-030123ebf43d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:26.290766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:26.309576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c6d94-da35-4c4c-874a-030123ebf43d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:26.341965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8494ca-2550-47c7-87cd-196dfc0f9f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:26.342548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8494ca-2550-47c7-87cd-196dfc0f9f1d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:26.342821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8494ca-2550-47c7-87cd-196dfc0f9f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:26.343084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8494ca-2550-47c7-87cd-196dfc0f9f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:26.343796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8494ca-2550-47c7-87cd-196dfc0f9f1d None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=005292fef03343c9a1dcb831e4d817e1, user.id=c303dffe4dfb4a0887849abd8a69b125) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:26.344097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8494ca-2550-47c7-87cd-196dfc0f9f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:26.345530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8494ca-2550-47c7-87cd-196dfc0f9f1d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:26.350642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 584/1185] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:26 2022] GET /identity/v3/role_assignments?user.id=c303dffe4dfb4a0887849abd8a69b125&scope.project.id=005292fef03343c9a1dcb831e4d817e1&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:28.211817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2025d-b0f4-4852-bf4d-1ec136eba18c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.212402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2025d-b0f4-4852-bf4d-1ec136eba18c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.212402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2025d-b0f4-4852-bf4d-1ec136eba18c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.213703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 602/1186] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:28.219018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0421e-8720-4df7-b47c-009b52d4fd2d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.219442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0421e-8720-4df7-b47c-009b52d4fd2d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.219554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0421e-8720-4df7-b47c-009b52d4fd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.237765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5fc0421e-8720-4df7-b47c-009b52d4fd2d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:28.238959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5fc0421e-8720-4df7-b47c-009b52d4fd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:28.311182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 585/1187] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:28.586513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2025d-b0f4-4852-bf4d-1ec136eba18c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.613436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.686952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2025d-b0f4-4852-bf4d-1ec136eba18c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.720830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743dc08-946c-4068-9d07-9984d51b437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.721483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743dc08-946c-4068-9d07-9984d51b437a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.721756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743dc08-946c-4068-9d07-9984d51b437a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.722020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743dc08-946c-4068-9d07-9984d51b437a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.732619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743dc08-946c-4068-9d07-9984d51b437a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-4743dc08-946c-4068-9d07-9984d51b437a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:28.737358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:28.738445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:50:28.739309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 603/1188] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:28.744454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0421e-8720-4df7-b47c-009b52d4fd2d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.819542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.842784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0421e-8720-4df7-b47c-009b52d4fd2d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.875474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdaadd1-3dcd-423f-8c9b-8fbb5ae34061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.876068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdaadd1-3dcd-423f-8c9b-8fbb5ae34061 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.876347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdaadd1-3dcd-423f-8c9b-8fbb5ae34061 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.876613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdaadd1-3dcd-423f-8c9b-8fbb5ae34061 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.877302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdaadd1-3dcd-423f-8c9b-8fbb5ae34061 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:28.877604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdaadd1-3dcd-423f-8c9b-8fbb5ae34061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:28.878945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdaadd1-3dcd-423f-8c9b-8fbb5ae34061 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:28.887452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 586/1189] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:28.893409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743dc08-946c-4068-9d07-9984d51b437a None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.978103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.992699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743dc08-946c-4068-9d07-9984d51b437a None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.026969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb713d-b84b-4564-9281-a214a287dc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.027599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb713d-b84b-4564-9281-a214a287dc76 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.027872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb713d-b84b-4564-9281-a214a287dc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.028138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb713d-b84b-4564-9281-a214a287dc76 None admin] PATH_INFO: `/v3/users/c303dffe4dfb4a0887849abd8a69b125` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.029263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb713d-b84b-4564-9281-a214a287dc76 None admin] RBAC: Authorizing `identity:get_user(user_id=c303dffe4dfb4a0887849abd8a69b125)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.030679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb713d-b84b-4564-9281-a214a287dc76 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.032226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 604/1190] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/users/c303dffe4dfb4a0887849abd8a69b125 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:29.038052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdaadd1-3dcd-423f-8c9b-8fbb5ae34061 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.138251 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdaadd1-3dcd-423f-8c9b-8fbb5ae34061 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.171822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a37b8-3ba4-4276-ae59-63645b0f0ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.172422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a37b8-3ba4-4276-ae59-63645b0f0ffa None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.172692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a37b8-3ba4-4276-ae59-63645b0f0ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.172957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a37b8-3ba4-4276-ae59-63645b0f0ffa None admin] PATH_INFO: `/v3/projects/005292fef03343c9a1dcb831e4d817e1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.174085 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a37b8-3ba4-4276-ae59-63645b0f0ffa None admin] RBAC: Authorizing `identity:get_project(project_id=005292fef03343c9a1dcb831e4d817e1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.175487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a37b8-3ba4-4276-ae59-63645b0f0ffa None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.177112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 587/1191] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/projects/005292fef03343c9a1dcb831e4d817e1 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:29.183169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb713d-b84b-4564-9281-a214a287dc76 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.281277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb713d-b84b-4564-9281-a214a287dc76 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.313971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d45e4-2103-4b25-a722-3cd2da3eca0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.314613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d45e4-2103-4b25-a722-3cd2da3eca0e None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.314890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d45e4-2103-4b25-a722-3cd2da3eca0e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.315158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d45e4-2103-4b25-a722-3cd2da3eca0e None admin] PATH_INFO: `/v3/projects/005292fef03343c9a1dcb831e4d817e1/users/c303dffe4dfb4a0887849abd8a69b125/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.317291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d45e4-2103-4b25-a722-3cd2da3eca0e None admin] RBAC: Authorizing `identity:create_grant(project_id=005292fef03343c9a1dcb831e4d817e1, user_id=c303dffe4dfb4a0887849abd8a69b125, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.318716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d45e4-2103-4b25-a722-3cd2da3eca0e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.331534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 605/1192] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:50:29 2022] PUT /identity/v3/projects/005292fef03343c9a1dcb831e4d817e1/users/c303dffe4dfb4a0887849abd8a69b125/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:50:31.197443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621025a-4218-423f-a734-50429234e7fe None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:31.197988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621025a-4218-423f-a734-50429234e7fe None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:31.197988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621025a-4218-423f-a734-50429234e7fe None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:31.199870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 588/1193] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:31.205039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012afc0c-072d-48b3-9ee1-c6fd93121a55 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:31.205495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012afc0c-072d-48b3-9ee1-c6fd93121a55 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:31.205614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012afc0c-072d-48b3-9ee1-c6fd93121a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:31.223881 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-012afc0c-072d-48b3-9ee1-c6fd93121a55 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:31.225054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-012afc0c-072d-48b3-9ee1-c6fd93121a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:31.234176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:31.293567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 606/1194] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:31.559570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621025a-4218-423f-a734-50429234e7fe None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:31.568936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:31.662202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621025a-4218-423f-a734-50429234e7fe None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:31.695559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07938aea-a12c-432f-a68d-170b638f7e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:31.696173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07938aea-a12c-432f-a68d-170b638f7e86 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:31.696451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07938aea-a12c-432f-a68d-170b638f7e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:31.696723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07938aea-a12c-432f-a68d-170b638f7e86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:31.707207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07938aea-a12c-432f-a68d-170b638f7e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-07938aea-a12c-432f-a68d-170b638f7e86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:31.711927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:31.713010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:50:31.714909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 589/1195] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:31.719617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012afc0c-072d-48b3-9ee1-c6fd93121a55 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:31.814542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012afc0c-072d-48b3-9ee1-c6fd93121a55 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:31.847163 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189aef2-2907-4d90-b548-5cf6bd0c98ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:31.847885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189aef2-2907-4d90-b548-5cf6bd0c98ef None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:31.848284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189aef2-2907-4d90-b548-5cf6bd0c98ef None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:31.848700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189aef2-2907-4d90-b548-5cf6bd0c98ef None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:31.849584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189aef2-2907-4d90-b548-5cf6bd0c98ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:31.849882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189aef2-2907-4d90-b548-5cf6bd0c98ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:31.851232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189aef2-2907-4d90-b548-5cf6bd0c98ef None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:31.860455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 607/1196] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:31.866029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07938aea-a12c-432f-a68d-170b638f7e86 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:31.929654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:31.964886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07938aea-a12c-432f-a68d-170b638f7e86 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:31.997574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2ce13-7b32-4e56-980e-202b52d43925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:31.998178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2ce13-7b32-4e56-980e-202b52d43925 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:31.998450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2ce13-7b32-4e56-980e-202b52d43925 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:31.998718 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2ce13-7b32-4e56-980e-202b52d43925 None admin] PATH_INFO: `/v3/users/c303dffe4dfb4a0887849abd8a69b125` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:31.999811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2ce13-7b32-4e56-980e-202b52d43925 None admin] RBAC: Authorizing `identity:get_user(user_id=c303dffe4dfb4a0887849abd8a69b125)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:32.001257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2ce13-7b32-4e56-980e-202b52d43925 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:32.002815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 590/1197] 10.222.0.78 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:31 2022] GET /identity/v3/users/c303dffe4dfb4a0887849abd8a69b125 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:32.008665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189aef2-2907-4d90-b548-5cf6bd0c98ef None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.018193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:32.108102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189aef2-2907-4d90-b548-5cf6bd0c98ef None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:32.140540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c8a16-6b09-4c35-a6b7-c0fbc84effdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:32.141144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c8a16-6b09-4c35-a6b7-c0fbc84effdb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.141423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c8a16-6b09-4c35-a6b7-c0fbc84effdb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.141687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c8a16-6b09-4c35-a6b7-c0fbc84effdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.208733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c8a16-6b09-4c35-a6b7-c0fbc84effdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:32.210208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c8a16-6b09-4c35-a6b7-c0fbc84effdb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:32.218286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:32.311289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 608/1198] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:32.317480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2ce13-7b32-4e56-980e-202b52d43925 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.413903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2ce13-7b32-4e56-980e-202b52d43925 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:32.454876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fcfe3-e6d4-49ad-9aa0-d8162b4882e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:32.455480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fcfe3-e6d4-49ad-9aa0-d8162b4882e9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.455754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fcfe3-e6d4-49ad-9aa0-d8162b4882e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.456021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fcfe3-e6d4-49ad-9aa0-d8162b4882e9 None admin] PATH_INFO: `/v3/projects/005292fef03343c9a1dcb831e4d817e1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.457136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fcfe3-e6d4-49ad-9aa0-d8162b4882e9 None admin] RBAC: Authorizing `identity:get_project(project_id=005292fef03343c9a1dcb831e4d817e1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:32.458549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fcfe3-e6d4-49ad-9aa0-d8162b4882e9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:32.460154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 591/1199] 10.222.0.78 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/projects/005292fef03343c9a1dcb831e4d817e1 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:32.466830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c8a16-6b09-4c35-a6b7-c0fbc84effdb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.564616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c8a16-6b09-4c35-a6b7-c0fbc84effdb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:32.572037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:32.598539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49ba8f-6937-4a50-93d6-a20655e588ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:32.599134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49ba8f-6937-4a50-93d6-a20655e588ec None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.599409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49ba8f-6937-4a50-93d6-a20655e588ec None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.599678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49ba8f-6937-4a50-93d6-a20655e588ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.600388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49ba8f-6937-4a50-93d6-a20655e588ec None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=005292fef03343c9a1dcb831e4d817e1, user.id=c303dffe4dfb4a0887849abd8a69b125) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:32.600691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49ba8f-6937-4a50-93d6-a20655e588ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:32.602138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49ba8f-6937-4a50-93d6-a20655e588ec None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:32.608257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 609/1200] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/role_assignments?user.id=c303dffe4dfb4a0887849abd8a69b125&scope.project.id=005292fef03343c9a1dcb831e4d817e1&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:42.442165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:42.605550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=15 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:28.511538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66fcf6-1140-477e-ba8d-222a306dea00 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:28.512139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66fcf6-1140-477e-ba8d-222a306dea00 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:28.512139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66fcf6-1140-477e-ba8d-222a306dea00 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:28.513636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 592/1201] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:51:28.520909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16970e8-3d2b-4995-a1d2-1d8426b0c0a5 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:28.520909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16970e8-3d2b-4995-a1d2-1d8426b0c0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:28.520909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16970e8-3d2b-4995-a1d2-1d8426b0c0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:28.544671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f16970e8-3d2b-4995-a1d2-1d8426b0c0a5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:28.545880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f16970e8-3d2b-4995-a1d2-1d8426b0c0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:28.631477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 610/1202] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:28.920576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66fcf6-1140-477e-ba8d-222a306dea00 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:29.067328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66fcf6-1140-477e-ba8d-222a306dea00 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:29.113809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709c783-0c3d-42a7-94a3-077b07ac4bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:29.114483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709c783-0c3d-42a7-94a3-077b07ac4bcb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:29.114768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709c783-0c3d-42a7-94a3-077b07ac4bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:29.115223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709c783-0c3d-42a7-94a3-077b07ac4bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:29.129175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709c783-0c3d-42a7-94a3-077b07ac4bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:29.130627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709c783-0c3d-42a7-94a3-077b07ac4bcb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-8709c783-0c3d-42a7-94a3-077b07ac4bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:29.149581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:29.150711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:29.151671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:29.151671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:29.151671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:29.151671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 593/1203] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:51:29.157616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16970e8-3d2b-4995-a1d2-1d8426b0c0a5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:29.294174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16970e8-3d2b-4995-a1d2-1d8426b0c0a5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:29.327329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:29.334756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3596f82-55ae-4625-b597-e5fbdb47097a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:29.335902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3596f82-55ae-4625-b597-e5fbdb47097a None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:29.336209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3596f82-55ae-4625-b597-e5fbdb47097a None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:29.336526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3596f82-55ae-4625-b597-e5fbdb47097a None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:29.337427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3596f82-55ae-4625-b597-e5fbdb47097a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:29.337735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3596f82-55ae-4625-b597-e5fbdb47097a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:29.339151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3596f82-55ae-4625-b597-e5fbdb47097a None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:29.351255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 611/1204] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:29.359153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709c783-0c3d-42a7-94a3-077b07ac4bcb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:29.422430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:29.479809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709c783-0c3d-42a7-94a3-077b07ac4bcb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:29.521109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575ce8b-8ddd-45d8-9e7b-a216cab28761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:29.521754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575ce8b-8ddd-45d8-9e7b-a216cab28761 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:29.521929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575ce8b-8ddd-45d8-9e7b-a216cab28761 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:29.522225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575ce8b-8ddd-45d8-9e7b-a216cab28761 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:29.522872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575ce8b-8ddd-45d8-9e7b-a216cab28761 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:29.524953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575ce8b-8ddd-45d8-9e7b-a216cab28761 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:29.550356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 594/1205] 10.222.0.78 () {60 vars in 1132 bytes} [Tue Aug 30 00:51:29 2022] POST /identity/v3/users => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:51:31.678849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb604ed3-f1f0-4a93-8217-7ed46cd70ab9 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:31.679402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb604ed3-f1f0-4a93-8217-7ed46cd70ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:31.679402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb604ed3-f1f0-4a93-8217-7ed46cd70ab9 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:31.681590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 612/1206] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:31.689973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652604c5-4292-48bd-918f-a2307cae03f4 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:31.690472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652604c5-4292-48bd-918f-a2307cae03f4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:31.690645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652604c5-4292-48bd-918f-a2307cae03f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:31.714032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-652604c5-4292-48bd-918f-a2307cae03f4 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:31.718009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-652604c5-4292-48bd-918f-a2307cae03f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:31.796334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 595/1207] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:32.118908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb604ed3-f1f0-4a93-8217-7ed46cd70ab9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:32.245537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb604ed3-f1f0-4a93-8217-7ed46cd70ab9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:32.290570 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15c226-6187-44e8-b86d-c2de2f7b12e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:32.291140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15c226-6187-44e8-b86d-c2de2f7b12e2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:32.291416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15c226-6187-44e8-b86d-c2de2f7b12e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:32.291681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15c226-6187-44e8-b86d-c2de2f7b12e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:32.302475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15c226-6187-44e8-b86d-c2de2f7b12e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-8a15c226-6187-44e8-b86d-c2de2f7b12e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:32.307859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:32.308832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:32.309747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 613/1208] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:51:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:51:32.320588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652604c5-4292-48bd-918f-a2307cae03f4 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:32.372194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:32.455052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652604c5-4292-48bd-918f-a2307cae03f4 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:32.495185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd891737-2903-4166-8616-9006aa3a53fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:32.495790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd891737-2903-4166-8616-9006aa3a53fb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:32.496066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd891737-2903-4166-8616-9006aa3a53fb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:32.496330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd891737-2903-4166-8616-9006aa3a53fb None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:32.496989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd891737-2903-4166-8616-9006aa3a53fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:32.497340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd891737-2903-4166-8616-9006aa3a53fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:32.498686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd891737-2903-4166-8616-9006aa3a53fb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:32.509908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 596/1209] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:51:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:32.516363 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15c226-6187-44e8-b86d-c2de2f7b12e2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:32.572909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:32.632905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15c226-6187-44e8-b86d-c2de2f7b12e2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:32.683075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dde266-18f9-4a10-a45c-66a494a27076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:32.685446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde266-18f9-4a10-a45c-66a494a27076 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:32.685446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde266-18f9-4a10-a45c-66a494a27076 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:32.685446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde266-18f9-4a10-a45c-66a494a27076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:32.697376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dde266-18f9-4a10-a45c-66a494a27076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:32.699011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dde266-18f9-4a10-a45c-66a494a27076 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-17dde266-18f9-4a10-a45c-66a494a27076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:32.718837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:32.719879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:32.720773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:32.720773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:32.720773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:32.727417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd891737-2903-4166-8616-9006aa3a53fb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:32.728187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 614/1210] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:32.843961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:32.848262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd891737-2903-4166-8616-9006aa3a53fb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:32.888867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ceadf1-f2ff-4b63-a6ed-e8ea1b9cde1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:32.889483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceadf1-f2ff-4b63-a6ed-e8ea1b9cde1b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:32.889764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceadf1-f2ff-4b63-a6ed-e8ea1b9cde1b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:32.890094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceadf1-f2ff-4b63-a6ed-e8ea1b9cde1b None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:32.890827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ceadf1-f2ff-4b63-a6ed-e8ea1b9cde1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:32.891151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ceadf1-f2ff-4b63-a6ed-e8ea1b9cde1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:32.892580 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ceadf1-f2ff-4b63-a6ed-e8ea1b9cde1b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:32.907335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 597/1211] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:32.913427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dde266-18f9-4a10-a45c-66a494a27076 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:33.040321 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dde266-18f9-4a10-a45c-66a494a27076 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:33.042164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:33.087443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a50380-6893-4614-8ea6-faa3417c9940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:33.088144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a50380-6893-4614-8ea6-faa3417c9940 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:33.088480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a50380-6893-4614-8ea6-faa3417c9940 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:33.088881 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a50380-6893-4614-8ea6-faa3417c9940 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:33.098412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a50380-6893-4614-8ea6-faa3417c9940 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:33.100322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a50380-6893-4614-8ea6-faa3417c9940 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-51a50380-6893-4614-8ea6-faa3417c9940 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:51:33.113406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:33.114464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:33.115428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 615/1212] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:33.122746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ceadf1-f2ff-4b63-a6ed-e8ea1b9cde1b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:33.257190 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ceadf1-f2ff-4b63-a6ed-e8ea1b9cde1b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:33.296022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236e6e0-11ef-4aca-b1a2-bde26057f3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:33.296639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236e6e0-11ef-4aca-b1a2-bde26057f3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:33.296951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236e6e0-11ef-4aca-b1a2-bde26057f3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:33.297256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236e6e0-11ef-4aca-b1a2-bde26057f3a2 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:33.298245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236e6e0-11ef-4aca-b1a2-bde26057f3a2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:33.298587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236e6e0-11ef-4aca-b1a2-bde26057f3a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:33.300266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236e6e0-11ef-4aca-b1a2-bde26057f3a2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:33.322714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 598/1213] 10.222.0.78 () {56 vars in 1143 bytes} [Tue Aug 30 00:51:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:33.328450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a50380-6893-4614-8ea6-faa3417c9940 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:33.466441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a50380-6893-4614-8ea6-faa3417c9940 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:33.507634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9f758-85de-4d9b-b7ca-cf21c45e24bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:33.509223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9f758-85de-4d9b-b7ca-cf21c45e24bd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:33.509523 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9f758-85de-4d9b-b7ca-cf21c45e24bd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:33.509802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9f758-85de-4d9b-b7ca-cf21c45e24bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:33.548387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:33.591906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9f758-85de-4d9b-b7ca-cf21c45e24bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:33.593569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9f758-85de-4d9b-b7ca-cf21c45e24bd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:33.726544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 616/1214] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:51:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:51:33.734468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236e6e0-11ef-4aca-b1a2-bde26057f3a2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:33.749155 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:33.845741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236e6e0-11ef-4aca-b1a2-bde26057f3a2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:33.888371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bfd23-9aa9-47ac-8480-6df66b1906a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:33.888994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bfd23-9aa9-47ac-8480-6df66b1906a7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:33.891722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bfd23-9aa9-47ac-8480-6df66b1906a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:33.891722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bfd23-9aa9-47ac-8480-6df66b1906a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:33.902214 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bfd23-9aa9-47ac-8480-6df66b1906a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:33.903732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bfd23-9aa9-47ac-8480-6df66b1906a7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-317bfd23-9aa9-47ac-8480-6df66b1906a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:33.919184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:33.920473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:33.921588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:33.921588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:33.921588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:33.926375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 599/1215] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:33.927569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9f758-85de-4d9b-b7ca-cf21c45e24bd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:33.991216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:34.035834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9f758-85de-4d9b-b7ca-cf21c45e24bd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:34.082899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cafa2-acb8-47ec-9f67-9fcd50a12d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:34.083933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cafa2-acb8-47ec-9f67-9fcd50a12d6b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:34.084284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cafa2-acb8-47ec-9f67-9fcd50a12d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:34.084630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cafa2-acb8-47ec-9f67-9fcd50a12d6b None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:34.085376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cafa2-acb8-47ec-9f67-9fcd50a12d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:34.085683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cafa2-acb8-47ec-9f67-9fcd50a12d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:34.087346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cafa2-acb8-47ec-9f67-9fcd50a12d6b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:34.102010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 617/1216] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:34.108275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bfd23-9aa9-47ac-8480-6df66b1906a7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:34.196311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:34.241422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bfd23-9aa9-47ac-8480-6df66b1906a7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:34.286557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080fa0b-7f47-4427-9235-1b32de42cbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:34.287559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080fa0b-7f47-4427-9235-1b32de42cbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:34.287882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080fa0b-7f47-4427-9235-1b32de42cbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:34.288338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080fa0b-7f47-4427-9235-1b32de42cbf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:34.296582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080fa0b-7f47-4427-9235-1b32de42cbf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:34.298088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080fa0b-7f47-4427-9235-1b32de42cbf1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c080fa0b-7f47-4427-9235-1b32de42cbf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:51:34.312416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:34.313606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:34.315106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:51:34.315106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:51:34.315106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:51:34.315106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:51:34.315106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:51:34.315106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:51:34.315106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:34.315106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:34.315106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 600/1217] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:51:34.320205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cafa2-acb8-47ec-9f67-9fcd50a12d6b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:34.441393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cafa2-acb8-47ec-9f67-9fcd50a12d6b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:34.469975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:34.484968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74135322-f5e6-419d-a6fb-5d4c49da89a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:34.485613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74135322-f5e6-419d-a6fb-5d4c49da89a7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:34.485892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74135322-f5e6-419d-a6fb-5d4c49da89a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:34.486171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74135322-f5e6-419d-a6fb-5d4c49da89a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:34.486852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74135322-f5e6-419d-a6fb-5d4c49da89a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:34.487182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74135322-f5e6-419d-a6fb-5d4c49da89a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:34.488582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74135322-f5e6-419d-a6fb-5d4c49da89a7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:34.505122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 618/1218] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:34.509993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080fa0b-7f47-4427-9235-1b32de42cbf1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:34.624913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080fa0b-7f47-4427-9235-1b32de42cbf1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:34.660364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:34.669222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b899a-9d9e-424b-9bcd-1f808e24ecd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:34.670015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b899a-9d9e-424b-9bcd-1f808e24ecd8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:34.670405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b899a-9d9e-424b-9bcd-1f808e24ecd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:34.670771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b899a-9d9e-424b-9bcd-1f808e24ecd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:34.671698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b899a-9d9e-424b-9bcd-1f808e24ecd8 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=8f860912e1f0406cadd70da440b2ede2) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:34.672067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b899a-9d9e-424b-9bcd-1f808e24ecd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:34.673756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b899a-9d9e-424b-9bcd-1f808e24ecd8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:34.680858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 601/1219] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:51:34 2022] GET /identity/v3/role_assignments?user.id=8f860912e1f0406cadd70da440b2ede2&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:36.928821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364c7dc-ac08-454b-a792-a3905506c805 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:36.929410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364c7dc-ac08-454b-a792-a3905506c805 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:36.929410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364c7dc-ac08-454b-a792-a3905506c805 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:36.931886 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 619/1220] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:36.937019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62024d2-bb27-4af2-b6f3-c3e7b96f45db None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:36.937461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62024d2-bb27-4af2-b6f3-c3e7b96f45db None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:36.937594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62024d2-bb27-4af2-b6f3-c3e7b96f45db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:36.957633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d62024d2-bb27-4af2-b6f3-c3e7b96f45db None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:36.958832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d62024d2-bb27-4af2-b6f3-c3e7b96f45db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:37.026917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 602/1221] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:37.324917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364c7dc-ac08-454b-a792-a3905506c805 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:37.429511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364c7dc-ac08-454b-a792-a3905506c805 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:37.464742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7939c0-3938-4bde-80f3-bece40e9f388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:37.465392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7939c0-3938-4bde-80f3-bece40e9f388 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:37.465666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7939c0-3938-4bde-80f3-bece40e9f388 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:37.465929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7939c0-3938-4bde-80f3-bece40e9f388 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:37.476435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7939c0-3938-4bde-80f3-bece40e9f388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-3b7939c0-3938-4bde-80f3-bece40e9f388 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:37.481926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:37.483285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:37.484166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 620/1222] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:51:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:37.489320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62024d2-bb27-4af2-b6f3-c3e7b96f45db None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:37.598028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62024d2-bb27-4af2-b6f3-c3e7b96f45db None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:37.624957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:37.637059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ace369-075c-4e9b-aec1-77972758ed70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:37.637651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace369-075c-4e9b-aec1-77972758ed70 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:37.637992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace369-075c-4e9b-aec1-77972758ed70 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:37.638232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace369-075c-4e9b-aec1-77972758ed70 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:37.639002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace369-075c-4e9b-aec1-77972758ed70 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:37.639339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace369-075c-4e9b-aec1-77972758ed70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:37.640785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace369-075c-4e9b-aec1-77972758ed70 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:37.650230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 603/1223] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:51:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:37.657745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7939c0-3938-4bde-80f3-bece40e9f388 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:37.710546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:37.785758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7939c0-3938-4bde-80f3-bece40e9f388 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:37.827212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55d2ca-a385-4b5d-86bb-a6ca8d0e73f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:37.827915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55d2ca-a385-4b5d-86bb-a6ca8d0e73f4 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:37.828195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55d2ca-a385-4b5d-86bb-a6ca8d0e73f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:37.828462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55d2ca-a385-4b5d-86bb-a6ca8d0e73f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:37.839742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55d2ca-a385-4b5d-86bb-a6ca8d0e73f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:37.841260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55d2ca-a385-4b5d-86bb-a6ca8d0e73f4 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-aa55d2ca-a385-4b5d-86bb-a6ca8d0e73f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:37.856908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:37.858108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:37.859158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:37.859158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:37.859158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:37.859158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 621/1224] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:37.863946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ace369-075c-4e9b-aec1-77972758ed70 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:37.976114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ace369-075c-4e9b-aec1-77972758ed70 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:38.016939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23defd1-4c18-4b3b-966e-7d1e5f6680be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:38.017570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23defd1-4c18-4b3b-966e-7d1e5f6680be None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:38.017846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23defd1-4c18-4b3b-966e-7d1e5f6680be None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:38.018267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23defd1-4c18-4b3b-966e-7d1e5f6680be None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:38.018964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23defd1-4c18-4b3b-966e-7d1e5f6680be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:38.019320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23defd1-4c18-4b3b-966e-7d1e5f6680be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:38.020770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23defd1-4c18-4b3b-966e-7d1e5f6680be None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:38.034342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 604/1225] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:38.041352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55d2ca-a385-4b5d-86bb-a6ca8d0e73f4 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:38.139888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:38.146879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55d2ca-a385-4b5d-86bb-a6ca8d0e73f4 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:38.182907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce157f-4028-4dc1-b5c9-b783289724dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:38.183528 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce157f-4028-4dc1-b5c9-b783289724dc None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:38.183801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce157f-4028-4dc1-b5c9-b783289724dc None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:38.184069 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce157f-4028-4dc1-b5c9-b783289724dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:38.192061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce157f-4028-4dc1-b5c9-b783289724dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:38.193620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce157f-4028-4dc1-b5c9-b783289724dc None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-abce157f-4028-4dc1-b5c9-b783289724dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:51:38.206222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:38.207432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:38.208807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 622/1226] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:38.213953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23defd1-4c18-4b3b-966e-7d1e5f6680be None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:38.265980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:38.325213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23defd1-4c18-4b3b-966e-7d1e5f6680be None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:38.360646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0605d6-3f36-43ba-b581-40afd0f644b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:38.361359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0605d6-3f36-43ba-b581-40afd0f644b1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:38.361643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0605d6-3f36-43ba-b581-40afd0f644b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:38.361930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0605d6-3f36-43ba-b581-40afd0f644b1 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:38.362760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0605d6-3f36-43ba-b581-40afd0f644b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:38.363124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0605d6-3f36-43ba-b581-40afd0f644b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:38.364683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0605d6-3f36-43ba-b581-40afd0f644b1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:38.383875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 605/1227] 10.222.0.78 () {56 vars in 1143 bytes} [Tue Aug 30 00:51:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:38.390437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce157f-4028-4dc1-b5c9-b783289724dc None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:38.497849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce157f-4028-4dc1-b5c9-b783289724dc None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:38.537146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa53147-f3c3-49fd-92c0-03bc08132f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:38.537777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa53147-f3c3-49fd-92c0-03bc08132f8b None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:38.538060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa53147-f3c3-49fd-92c0-03bc08132f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:38.538332 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa53147-f3c3-49fd-92c0-03bc08132f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:38.551045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa53147-f3c3-49fd-92c0-03bc08132f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:38.552513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa53147-f3c3-49fd-92c0-03bc08132f8b None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-afa53147-f3c3-49fd-92c0-03bc08132f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:38.568540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:38.569789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:38.571701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:38.571701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:38.571701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:38.571701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 623/1228] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:38.576955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0605d6-3f36-43ba-b581-40afd0f644b1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:38.697449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0605d6-3f36-43ba-b581-40afd0f644b1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:38.701111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:38.747087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d079e3-bf02-42d1-a725-19ca93800c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:38.747693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d079e3-bf02-42d1-a725-19ca93800c1a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:38.747970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d079e3-bf02-42d1-a725-19ca93800c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:38.748237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d079e3-bf02-42d1-a725-19ca93800c1a None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:38.748877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d079e3-bf02-42d1-a725-19ca93800c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:38.749411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d079e3-bf02-42d1-a725-19ca93800c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:38.750573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d079e3-bf02-42d1-a725-19ca93800c1a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:38.762923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 606/1229] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:38.769005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa53147-f3c3-49fd-92c0-03bc08132f8b None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:38.780023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:38.878893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa53147-f3c3-49fd-92c0-03bc08132f8b None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:38.916762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c6369-fd4e-40eb-a33b-e5335b287078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:38.917511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c6369-fd4e-40eb-a33b-e5335b287078 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:38.917791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c6369-fd4e-40eb-a33b-e5335b287078 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:38.918066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c6369-fd4e-40eb-a33b-e5335b287078 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:38.925210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c6369-fd4e-40eb-a33b-e5335b287078 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:38.926656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c6369-fd4e-40eb-a33b-e5335b287078 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-da6c6369-fd4e-40eb-a33b-e5335b287078 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:51:38.939094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:38.945662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:38.946800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:51:38.946800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:51:38.946800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:51:38.946800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:51:38.946800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:51:38.946800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:51:38.946800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:38.946800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:38.946800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 624/1230] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:38.948496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d079e3-bf02-42d1-a725-19ca93800c1a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:39.055176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d079e3-bf02-42d1-a725-19ca93800c1a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:39.098579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ae3b7-120e-430d-922a-c8ded0fe0c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:39.099206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ae3b7-120e-430d-922a-c8ded0fe0c98 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:39.099481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ae3b7-120e-430d-922a-c8ded0fe0c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:39.099759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ae3b7-120e-430d-922a-c8ded0fe0c98 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:39.100457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ae3b7-120e-430d-922a-c8ded0fe0c98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:39.100772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ae3b7-120e-430d-922a-c8ded0fe0c98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:39.102547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ae3b7-120e-430d-922a-c8ded0fe0c98 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:39.114922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 607/1231] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:51:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:39.122456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c6369-fd4e-40eb-a33b-e5335b287078 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:39.191645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:39.232189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c6369-fd4e-40eb-a33b-e5335b287078 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:39.268989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab897a-8a2a-4f64-9cc4-b87bad7568fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:39.269690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab897a-8a2a-4f64-9cc4-b87bad7568fe None admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:39.269961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab897a-8a2a-4f64-9cc4-b87bad7568fe None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:39.270240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab897a-8a2a-4f64-9cc4-b87bad7568fe None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/8f860912e1f0406cadd70da440b2ede2/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:39.287631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab897a-8a2a-4f64-9cc4-b87bad7568fe None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=8f860912e1f0406cadd70da440b2ede2, role_id=fd7dbf5fe1ae4cdf8a10f67c6921d3da)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:39.289247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab897a-8a2a-4f64-9cc4-b87bad7568fe None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:39.302485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 625/1232] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:51:39 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/8f860912e1f0406cadd70da440b2ede2/roles/fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:51:41.415074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16f18c-6e86-43e5-afd0-d39002fc46ef None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:41.415622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16f18c-6e86-43e5-afd0-d39002fc46ef None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:41.415622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16f18c-6e86-43e5-afd0-d39002fc46ef None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:41.417239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 608/1233] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:41.423133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811c22d-31cc-4437-be77-399862f4c246 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:41.423579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811c22d-31cc-4437-be77-399862f4c246 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:41.423672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811c22d-31cc-4437-be77-399862f4c246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:41.443653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1811c22d-31cc-4437-be77-399862f4c246 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:41.444931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1811c22d-31cc-4437-be77-399862f4c246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:41.515882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 626/1234] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:41.821013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16f18c-6e86-43e5-afd0-d39002fc46ef None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:41.849474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:41.938224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16f18c-6e86-43e5-afd0-d39002fc46ef None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:41.974475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec4fe-ccf0-4af8-b3e4-86d027dfe477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:41.975031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec4fe-ccf0-4af8-b3e4-86d027dfe477 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:41.975323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec4fe-ccf0-4af8-b3e4-86d027dfe477 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:41.975581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec4fe-ccf0-4af8-b3e4-86d027dfe477 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:41.986433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ec4fe-ccf0-4af8-b3e4-86d027dfe477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-cf8ec4fe-ccf0-4af8-b3e4-86d027dfe477 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:41.991260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:41.992686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:51:41.993851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:41.995430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 609/1235] 10.222.0.78 () {56 vars in 1102 bytes} [Tue Aug 30 00:51:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:42.001587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811c22d-31cc-4437-be77-399862f4c246 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:42.053453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:42.114912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811c22d-31cc-4437-be77-399862f4c246 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:42.155300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c85df3-1bf9-40e1-a54d-9899937179fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:42.155964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c85df3-1bf9-40e1-a54d-9899937179fb None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:42.156242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c85df3-1bf9-40e1-a54d-9899937179fb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:42.156514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c85df3-1bf9-40e1-a54d-9899937179fb None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:42.157253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c85df3-1bf9-40e1-a54d-9899937179fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:42.157630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c85df3-1bf9-40e1-a54d-9899937179fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:42.159260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c85df3-1bf9-40e1-a54d-9899937179fb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:42.168569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 627/1236] 10.222.0.78 () {56 vars in 1103 bytes} [Tue Aug 30 00:51:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:42.174467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec4fe-ccf0-4af8-b3e4-86d027dfe477 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:42.280055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:42.298308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec4fe-ccf0-4af8-b3e4-86d027dfe477 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:42.337328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3a166-c39a-400f-b157-80be887de7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:42.337953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3a166-c39a-400f-b157-80be887de7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:42.338236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3a166-c39a-400f-b157-80be887de7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:42.338500 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3a166-c39a-400f-b157-80be887de7c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:42.348614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3a166-c39a-400f-b157-80be887de7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:42.350086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3a166-c39a-400f-b157-80be887de7c3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-92c3a166-c39a-400f-b157-80be887de7c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:42.365566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:42.366711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:42.367974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:42.367974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:42.367974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:42.369274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 610/1237] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:42.373314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c85df3-1bf9-40e1-a54d-9899937179fb None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:42.484964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c85df3-1bf9-40e1-a54d-9899937179fb None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:42.495641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:42.524354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfcf5f-0284-495b-b169-90e536831ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:42.525182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfcf5f-0284-495b-b169-90e536831ece None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:42.525493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfcf5f-0284-495b-b169-90e536831ece None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:42.525768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfcf5f-0284-495b-b169-90e536831ece None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:42.526480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfcf5f-0284-495b-b169-90e536831ece None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:42.526897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfcf5f-0284-495b-b169-90e536831ece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:42.528552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfcf5f-0284-495b-b169-90e536831ece None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:42.542462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 628/1238] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:42.549245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3a166-c39a-400f-b157-80be887de7c3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:42.667589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3a166-c39a-400f-b157-80be887de7c3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:42.709744 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19c47b-49be-47b3-950c-f4c69831ff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:42.710553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19c47b-49be-47b3-950c-f4c69831ff23 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:42.711798 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19c47b-49be-47b3-950c-f4c69831ff23 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:42.712261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19c47b-49be-47b3-950c-f4c69831ff23 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:42.718282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:42.725165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19c47b-49be-47b3-950c-f4c69831ff23 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:42.727779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19c47b-49be-47b3-950c-f4c69831ff23 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-cf19c47b-49be-47b3-950c-f4c69831ff23 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:51:42.743144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:42.744310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:42.745570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 611/1239] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:42.750844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfcf5f-0284-495b-b169-90e536831ece None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:42.863376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfcf5f-0284-495b-b169-90e536831ece None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:42.905647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac54fe5-bee7-4ff1-ad01-5e8772cffbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:42.906399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac54fe5-bee7-4ff1-ad01-5e8772cffbc7 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:42.906778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac54fe5-bee7-4ff1-ad01-5e8772cffbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:42.907149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac54fe5-bee7-4ff1-ad01-5e8772cffbc7 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:42.908166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac54fe5-bee7-4ff1-ad01-5e8772cffbc7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:42.908574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac54fe5-bee7-4ff1-ad01-5e8772cffbc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:42.910126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac54fe5-bee7-4ff1-ad01-5e8772cffbc7 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:42.927687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 629/1240] 10.222.0.78 () {56 vars in 1143 bytes} [Tue Aug 30 00:51:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:51:42.934247 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19c47b-49be-47b3-950c-f4c69831ff23 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:43.040015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19c47b-49be-47b3-950c-f4c69831ff23 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:43.078050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071afaad-b90a-4a9f-be6b-bc530abd2ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:43.078667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071afaad-b90a-4a9f-be6b-bc530abd2ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:43.078929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071afaad-b90a-4a9f-be6b-bc530abd2ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:43.079199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071afaad-b90a-4a9f-be6b-bc530abd2ed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:43.149961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:43.156734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071afaad-b90a-4a9f-be6b-bc530abd2ed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:43.158220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071afaad-b90a-4a9f-be6b-bc530abd2ed9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:43.272504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 612/1241] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:51:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:51:43.278309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac54fe5-bee7-4ff1-ad01-5e8772cffbc7 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:43.308318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:43.390506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac54fe5-bee7-4ff1-ad01-5e8772cffbc7 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:43.429174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ff04a-65bd-4a51-9126-29f895dc31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:43.430416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ff04a-65bd-4a51-9126-29f895dc31ef None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:43.430691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ff04a-65bd-4a51-9126-29f895dc31ef None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:43.430957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ff04a-65bd-4a51-9126-29f895dc31ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:43.441562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ff04a-65bd-4a51-9126-29f895dc31ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:43.443143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ff04a-65bd-4a51-9126-29f895dc31ef None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-b98ff04a-65bd-4a51-9126-29f895dc31ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:43.459181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:43.460453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:43.461747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:43.461747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:43.461747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:43.461747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 630/1242] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:43.466123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071afaad-b90a-4a9f-be6b-bc530abd2ed9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:43.567264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:43.574049 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071afaad-b90a-4a9f-be6b-bc530abd2ed9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:43.612501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6a7aa-06de-46db-bc5f-245c95875ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:43.613129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6a7aa-06de-46db-bc5f-245c95875ccf None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:43.613416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6a7aa-06de-46db-bc5f-245c95875ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:43.613676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6a7aa-06de-46db-bc5f-245c95875ccf None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:43.614352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6a7aa-06de-46db-bc5f-245c95875ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:43.614609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6a7aa-06de-46db-bc5f-245c95875ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:43.615964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6a7aa-06de-46db-bc5f-245c95875ccf None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:43.634126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 613/1243] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:51:43.635425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ff04a-65bd-4a51-9126-29f895dc31ef None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:43.727737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:43.753532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ff04a-65bd-4a51-9126-29f895dc31ef None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:43.788299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca23d7-b700-4c41-8bef-9f00b5af4f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:43.790103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca23d7-b700-4c41-8bef-9f00b5af4f00 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:43.790380 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca23d7-b700-4c41-8bef-9f00b5af4f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:43.790646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca23d7-b700-4c41-8bef-9f00b5af4f00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:43.797652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca23d7-b700-4c41-8bef-9f00b5af4f00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:43.799171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca23d7-b700-4c41-8bef-9f00b5af4f00 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-98ca23d7-b700-4c41-8bef-9f00b5af4f00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:51:43.812482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:43.813657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:43.815023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:51:43.815023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:51:43.815023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:51:43.815023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:51:43.815023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:51:43.815023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:51:43.815023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:43.815023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:43.815023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 631/1244] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:43.820090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6a7aa-06de-46db-bc5f-245c95875ccf None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:43.931886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6a7aa-06de-46db-bc5f-245c95875ccf None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:43.967286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae21c57-2d7b-424e-8ae3-c25113cc7122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:43.967890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae21c57-2d7b-424e-8ae3-c25113cc7122 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:43.968167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae21c57-2d7b-424e-8ae3-c25113cc7122 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:43.968438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae21c57-2d7b-424e-8ae3-c25113cc7122 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:43.969940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae21c57-2d7b-424e-8ae3-c25113cc7122 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:43.969940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae21c57-2d7b-424e-8ae3-c25113cc7122 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:43.970871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae21c57-2d7b-424e-8ae3-c25113cc7122 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:43.984761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 614/1245] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:51:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:51:43.990971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca23d7-b700-4c41-8bef-9f00b5af4f00 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:44.103488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca23d7-b700-4c41-8bef-9f00b5af4f00 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:44.139803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac870137-5ad5-485e-8787-92f34b68ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:44.140410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac870137-5ad5-485e-8787-92f34b68ee63 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:44.140690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac870137-5ad5-485e-8787-92f34b68ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:44.140962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac870137-5ad5-485e-8787-92f34b68ee63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:44.141736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac870137-5ad5-485e-8787-92f34b68ee63 None admin] RBAC: Adding query filter params (role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=8f860912e1f0406cadd70da440b2ede2) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:44.142038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac870137-5ad5-485e-8787-92f34b68ee63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:44.143469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac870137-5ad5-485e-8787-92f34b68ee63 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:44.150276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 632/1246] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:51:43 2022] GET /identity/v3/role_assignments?user.id=8f860912e1f0406cadd70da440b2ede2&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=fd7dbf5fe1ae4cdf8a10f67c6921d3da => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:46.285865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06541aa-6ef7-4821-a721-cf8a778e8a6c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:46.286410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06541aa-6ef7-4821-a721-cf8a778e8a6c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:46.286410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06541aa-6ef7-4821-a721-cf8a778e8a6c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:46.287740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 615/1247] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:46.294826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715c7bf-bf23-49d3-8c98-7c67a3f7edbf None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:46.295327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715c7bf-bf23-49d3-8c98-7c67a3f7edbf None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:46.295422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715c7bf-bf23-49d3-8c98-7c67a3f7edbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:46.302245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:46.316308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d715c7bf-bf23-49d3-8c98-7c67a3f7edbf None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:46.317559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d715c7bf-bf23-49d3-8c98-7c67a3f7edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:46.398134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 633/1248] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:46.702478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06541aa-6ef7-4821-a721-cf8a778e8a6c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:46.725878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:46.825011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06541aa-6ef7-4821-a721-cf8a778e8a6c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:46.864993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebef6d-18e6-487b-877b-28a1a6a109ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:46.865667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebef6d-18e6-487b-877b-28a1a6a109ac None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:46.865925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebef6d-18e6-487b-877b-28a1a6a109ac None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:46.866189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebef6d-18e6-487b-877b-28a1a6a109ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:46.880491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfebef6d-18e6-487b-877b-28a1a6a109ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-cfebef6d-18e6-487b-877b-28a1a6a109ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:46.885772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:46.886933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:46.888147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 616/1249] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:51:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:46.892834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715c7bf-bf23-49d3-8c98-7c67a3f7edbf None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:46.999750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715c7bf-bf23-49d3-8c98-7c67a3f7edbf None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:47.037158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:47.043264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362e37a-a883-4bde-8e7a-321cc5d24fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:47.043919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362e37a-a883-4bde-8e7a-321cc5d24fbf None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:47.044209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362e37a-a883-4bde-8e7a-321cc5d24fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:47.044533 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362e37a-a883-4bde-8e7a-321cc5d24fbf None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:47.045284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362e37a-a883-4bde-8e7a-321cc5d24fbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:47.045591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362e37a-a883-4bde-8e7a-321cc5d24fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:47.047153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362e37a-a883-4bde-8e7a-321cc5d24fbf None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:47.056171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 634/1250] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:51:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:47.062122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebef6d-18e6-487b-877b-28a1a6a109ac None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:47.148232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:47.181819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebef6d-18e6-487b-877b-28a1a6a109ac None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:47.218995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137ff02-10e6-466d-8c19-2f4af0b27e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:47.219620 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137ff02-10e6-466d-8c19-2f4af0b27e3a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:47.219904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137ff02-10e6-466d-8c19-2f4af0b27e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:47.220184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137ff02-10e6-466d-8c19-2f4af0b27e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:47.230834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137ff02-10e6-466d-8c19-2f4af0b27e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:47.232264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137ff02-10e6-466d-8c19-2f4af0b27e3a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-5137ff02-10e6-466d-8c19-2f4af0b27e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:47.248612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:47.249887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:47.251019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:47.251019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:47.251019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:47.251019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 617/1251] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:47.256017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362e37a-a883-4bde-8e7a-321cc5d24fbf None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:47.361826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362e37a-a883-4bde-8e7a-321cc5d24fbf None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:47.399891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95af642-1e55-42b7-a6ef-5e781400d854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:47.400497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95af642-1e55-42b7-a6ef-5e781400d854 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:47.400776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95af642-1e55-42b7-a6ef-5e781400d854 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:47.401048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95af642-1e55-42b7-a6ef-5e781400d854 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:47.401709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95af642-1e55-42b7-a6ef-5e781400d854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:47.402009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95af642-1e55-42b7-a6ef-5e781400d854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:47.403370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95af642-1e55-42b7-a6ef-5e781400d854 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:47.416777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 635/1252] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:47.423166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137ff02-10e6-466d-8c19-2f4af0b27e3a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:47.543110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137ff02-10e6-466d-8c19-2f4af0b27e3a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:47.572741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:47.580831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c2947-c125-48c2-9f15-3f59268f53e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:47.581477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c2947-c125-48c2-9f15-3f59268f53e4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:47.581758 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c2947-c125-48c2-9f15-3f59268f53e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:47.582028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c2947-c125-48c2-9f15-3f59268f53e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:47.589644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c2947-c125-48c2-9f15-3f59268f53e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:47.591058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c2947-c125-48c2-9f15-3f59268f53e4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3f1c2947-c125-48c2-9f15-3f59268f53e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:51:47.602438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:47.603595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:47.605140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 618/1253] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:47.609453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95af642-1e55-42b7-a6ef-5e781400d854 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:47.671648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:47.725239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95af642-1e55-42b7-a6ef-5e781400d854 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:47.775263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4cd57-089e-4877-ba8d-e55d53e6379c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:47.776116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4cd57-089e-4877-ba8d-e55d53e6379c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:47.776492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4cd57-089e-4877-ba8d-e55d53e6379c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:47.776868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4cd57-089e-4877-ba8d-e55d53e6379c None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:47.778013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4cd57-089e-4877-ba8d-e55d53e6379c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:47.778402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4cd57-089e-4877-ba8d-e55d53e6379c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:47.779820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4cd57-089e-4877-ba8d-e55d53e6379c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:47.802078 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 636/1254] 10.222.0.78 () {56 vars in 1143 bytes} [Tue Aug 30 00:51:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:47.803129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c2947-c125-48c2-9f15-3f59268f53e4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:47.935061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c2947-c125-48c2-9f15-3f59268f53e4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:47.971037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38d81-5f9c-447f-bbf1-c630f8acb572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:47.971701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38d81-5f9c-447f-bbf1-c630f8acb572 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:47.971935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38d81-5f9c-447f-bbf1-c630f8acb572 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:47.972201 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38d81-5f9c-447f-bbf1-c630f8acb572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:48.034115 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:48.049964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda38d81-5f9c-447f-bbf1-c630f8acb572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:48.051457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda38d81-5f9c-447f-bbf1-c630f8acb572 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:48.160629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 619/1255] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:51:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:51:48.166662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4cd57-089e-4877-ba8d-e55d53e6379c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:48.285289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4cd57-089e-4877-ba8d-e55d53e6379c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:48.299922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:48.325469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a31df-1308-400f-a36b-95061eb1f737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:48.326229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a31df-1308-400f-a36b-95061eb1f737 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:48.326533 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a31df-1308-400f-a36b-95061eb1f737 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:48.326844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a31df-1308-400f-a36b-95061eb1f737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:48.337816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a31df-1308-400f-a36b-95061eb1f737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:48.339408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a31df-1308-400f-a36b-95061eb1f737 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-239a31df-1308-400f-a36b-95061eb1f737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:48.357307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:48.358530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:48.359447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:48.359447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:48.359447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:48.359447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 637/1256] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:48.364527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38d81-5f9c-447f-bbf1-c630f8acb572 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:48.469438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:48.483132 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38d81-5f9c-447f-bbf1-c630f8acb572 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:48.519565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6364cc-0f45-4a47-a25a-7263f97e8d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:48.520183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6364cc-0f45-4a47-a25a-7263f97e8d02 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:48.520461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6364cc-0f45-4a47-a25a-7263f97e8d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:48.520732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6364cc-0f45-4a47-a25a-7263f97e8d02 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:48.521425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6364cc-0f45-4a47-a25a-7263f97e8d02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:48.521729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6364cc-0f45-4a47-a25a-7263f97e8d02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:48.524061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6364cc-0f45-4a47-a25a-7263f97e8d02 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:48.536919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 620/1257] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:51:48.542796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a31df-1308-400f-a36b-95061eb1f737 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:48.650881 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a31df-1308-400f-a36b-95061eb1f737 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:48.696473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf63acd-6e9b-4d40-8780-3046ee5e0041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:48.697224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf63acd-6e9b-4d40-8780-3046ee5e0041 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:48.697522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf63acd-6e9b-4d40-8780-3046ee5e0041 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:48.697872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf63acd-6e9b-4d40-8780-3046ee5e0041 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:48.706735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf63acd-6e9b-4d40-8780-3046ee5e0041 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:48.708743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf63acd-6e9b-4d40-8780-3046ee5e0041 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-7bf63acd-6e9b-4d40-8780-3046ee5e0041 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:51:48.723532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:48.724683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:48.732355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:51:48.732355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:51:48.732355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:51:48.732355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:51:48.732355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:51:48.732355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:51:48.732355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:48.732355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:48.732355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 638/1258] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:51:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:48.733219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6364cc-0f45-4a47-a25a-7263f97e8d02 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:48.847533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6364cc-0f45-4a47-a25a-7263f97e8d02 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:48.882098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0953181-b0c2-471e-ad06-17b37949c62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:48.882833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0953181-b0c2-471e-ad06-17b37949c62d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:48.883112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0953181-b0c2-471e-ad06-17b37949c62d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:48.883384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0953181-b0c2-471e-ad06-17b37949c62d None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:48.884101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0953181-b0c2-471e-ad06-17b37949c62d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:48.884406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0953181-b0c2-471e-ad06-17b37949c62d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:48.885848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0953181-b0c2-471e-ad06-17b37949c62d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:48.899052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 621/1259] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:51:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:51:48.904816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf63acd-6e9b-4d40-8780-3046ee5e0041 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:48.931885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:49.011009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf63acd-6e9b-4d40-8780-3046ee5e0041 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:49.044855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edb72c-55c8-4b23-a964-c15b8f16db16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:49.045531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edb72c-55c8-4b23-a964-c15b8f16db16 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:49.045812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edb72c-55c8-4b23-a964-c15b8f16db16 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:49.046088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edb72c-55c8-4b23-a964-c15b8f16db16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:49.046806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edb72c-55c8-4b23-a964-c15b8f16db16 None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=8f860912e1f0406cadd70da440b2ede2) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:49.047114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edb72c-55c8-4b23-a964-c15b8f16db16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:49.048543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edb72c-55c8-4b23-a964-c15b8f16db16 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:49.054074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 639/1260] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:51:48 2022] GET /identity/v3/role_assignments?user.id=8f860912e1f0406cadd70da440b2ede2&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:50.964313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027a7f4-ed36-41d5-80e2-7f503010178c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:50.964945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027a7f4-ed36-41d5-80e2-7f503010178c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:50.964945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027a7f4-ed36-41d5-80e2-7f503010178c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:50.966221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 622/1261] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:50.971316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b1505-8785-4c42-aafa-b988a6b7e042 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:50.971760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b1505-8785-4c42-aafa-b988a6b7e042 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:50.971858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b1505-8785-4c42-aafa-b988a6b7e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:50.989477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a60b1505-8785-4c42-aafa-b988a6b7e042 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:50.990642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a60b1505-8785-4c42-aafa-b988a6b7e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:51.054590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 640/1262] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:51.328370 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027a7f4-ed36-41d5-80e2-7f503010178c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:51.339266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:51.428093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027a7f4-ed36-41d5-80e2-7f503010178c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:51.461898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7256285-fbcb-4c30-a5ee-b75e4f187445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:51.462505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7256285-fbcb-4c30-a5ee-b75e4f187445 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:51.462784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7256285-fbcb-4c30-a5ee-b75e4f187445 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:51.463055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7256285-fbcb-4c30-a5ee-b75e4f187445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:51.472924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7256285-fbcb-4c30-a5ee-b75e4f187445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a7256285-fbcb-4c30-a5ee-b75e4f187445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:51.478402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:51.479425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:51.480415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 623/1263] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:51:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:51.485471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b1505-8785-4c42-aafa-b988a6b7e042 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:51.503430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:51.581057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b1505-8785-4c42-aafa-b988a6b7e042 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:51.613576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3596c-a9d0-46e9-b269-e237d6f85b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:51.614165 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3596c-a9d0-46e9-b269-e237d6f85b1e None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:51.614440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3596c-a9d0-46e9-b269-e237d6f85b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:51.614705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3596c-a9d0-46e9-b269-e237d6f85b1e None admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:51.615351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3596c-a9d0-46e9-b269-e237d6f85b1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:51.615645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3596c-a9d0-46e9-b269-e237d6f85b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:51.617018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3596c-a9d0-46e9-b269-e237d6f85b1e None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:51.624670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 641/1264] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:51:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:51.630345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7256285-fbcb-4c30-a5ee-b75e4f187445 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:51.699983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:51.744329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7256285-fbcb-4c30-a5ee-b75e4f187445 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:51.783041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ce9c5-90ed-49bd-9830-d990d31d8d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:51.783655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ce9c5-90ed-49bd-9830-d990d31d8d9e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:51.783928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ce9c5-90ed-49bd-9830-d990d31d8d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:51.784194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ce9c5-90ed-49bd-9830-d990d31d8d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:51.794978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ce9c5-90ed-49bd-9830-d990d31d8d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:51.796414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ce9c5-90ed-49bd-9830-d990d31d8d9e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-132ce9c5-90ed-49bd-9830-d990d31d8d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:51.810013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:51.810932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:51.811847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:51.811847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:51.811847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:51.811847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 624/1265] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:51.817171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3596c-a9d0-46e9-b269-e237d6f85b1e None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:51.903505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:51.922671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3596c-a9d0-46e9-b269-e237d6f85b1e None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:51.955100 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e11778-d224-46f4-b077-dc22fa70d4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:51.955693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e11778-d224-46f4-b077-dc22fa70d4ac None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:51.955996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e11778-d224-46f4-b077-dc22fa70d4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:51.956267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e11778-d224-46f4-b077-dc22fa70d4ac None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:51.956892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e11778-d224-46f4-b077-dc22fa70d4ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:51.957221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e11778-d224-46f4-b077-dc22fa70d4ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:51.958589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e11778-d224-46f4-b077-dc22fa70d4ac None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:51.970355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 642/1266] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:51.976108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ce9c5-90ed-49bd-9830-d990d31d8d9e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:52.073367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ce9c5-90ed-49bd-9830-d990d31d8d9e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:52.092755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:52.107060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e683c-458d-4415-9673-0e52fb0e9086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:52.107673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e683c-458d-4415-9673-0e52fb0e9086 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:52.107951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e683c-458d-4415-9673-0e52fb0e9086 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:52.108224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e683c-458d-4415-9673-0e52fb0e9086 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:52.115434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e683c-458d-4415-9673-0e52fb0e9086 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:52.116987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e683c-458d-4415-9673-0e52fb0e9086 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-902e683c-458d-4415-9673-0e52fb0e9086 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:51:52.128145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:52.129295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:52.130429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 625/1267] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:52.135804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e11778-d224-46f4-b077-dc22fa70d4ac None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:52.236386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e11778-d224-46f4-b077-dc22fa70d4ac None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:52.269707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac500b-e8d3-49c2-a6cc-0bb74dc303b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:52.270301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac500b-e8d3-49c2-a6cc-0bb74dc303b9 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:52.270572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac500b-e8d3-49c2-a6cc-0bb74dc303b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:52.270836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac500b-e8d3-49c2-a6cc-0bb74dc303b9 None admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:52.271543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac500b-e8d3-49c2-a6cc-0bb74dc303b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:52.271840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac500b-e8d3-49c2-a6cc-0bb74dc303b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:52.273848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac500b-e8d3-49c2-a6cc-0bb74dc303b9 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:52.282370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:52.291087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 643/1268] 10.222.0.78 () {56 vars in 1143 bytes} [Tue Aug 30 00:51:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:52.296751 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e683c-458d-4415-9673-0e52fb0e9086 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:52.394631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e683c-458d-4415-9673-0e52fb0e9086 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:52.428364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f871175-0f89-4f12-b930-63467437dbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:52.429628 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f871175-0f89-4f12-b930-63467437dbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:52.429912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f871175-0f89-4f12-b930-63467437dbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:52.430186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f871175-0f89-4f12-b930-63467437dbd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:52.440686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f871175-0f89-4f12-b930-63467437dbd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:52.442162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f871175-0f89-4f12-b930-63467437dbd0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7f871175-0f89-4f12-b930-63467437dbd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:52.456491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:52.457626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:52.458850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:52.458850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:52.458850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:52.458850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 626/1269] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:52.463696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac500b-e8d3-49c2-a6cc-0bb74dc303b9 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:52.561694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac500b-e8d3-49c2-a6cc-0bb74dc303b9 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:52.595259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c5203-d2ef-40c1-9906-de8a3b33f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:52.595853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c5203-d2ef-40c1-9906-de8a3b33f9ff None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:52.596132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c5203-d2ef-40c1-9906-de8a3b33f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:52.596400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c5203-d2ef-40c1-9906-de8a3b33f9ff None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:52.597027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c5203-d2ef-40c1-9906-de8a3b33f9ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:52.597355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c5203-d2ef-40c1-9906-de8a3b33f9ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:52.598705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c5203-d2ef-40c1-9906-de8a3b33f9ff None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:52.609571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 644/1270] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:52.615285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f871175-0f89-4f12-b930-63467437dbd0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:52.646942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:52.719932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f871175-0f89-4f12-b930-63467437dbd0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:52.755307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8841281-2c76-4ed7-950e-5dbde6c90fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:52.755926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8841281-2c76-4ed7-950e-5dbde6c90fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:52.756196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8841281-2c76-4ed7-950e-5dbde6c90fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:52.756467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8841281-2c76-4ed7-950e-5dbde6c90fe3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:52.763798 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8841281-2c76-4ed7-950e-5dbde6c90fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:52.765259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8841281-2c76-4ed7-950e-5dbde6c90fe3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-e8841281-2c76-4ed7-950e-5dbde6c90fe3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:51:52.776661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:52.778363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:52.780373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:51:52.780373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:51:52.780373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:51:52.780373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:51:52.780373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:51:52.780373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:51:52.780373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:52.780373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:52.780373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 627/1271] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:51:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:52.784974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c5203-d2ef-40c1-9906-de8a3b33f9ff None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:52.844519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:52.887409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c5203-d2ef-40c1-9906-de8a3b33f9ff None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:52.921251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd689e-15f0-4835-963c-b871f3ee2a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:52.921843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd689e-15f0-4835-963c-b871f3ee2a2f None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:52.922124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd689e-15f0-4835-963c-b871f3ee2a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:52.922395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd689e-15f0-4835-963c-b871f3ee2a2f None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:52.923074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd689e-15f0-4835-963c-b871f3ee2a2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:52.923376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd689e-15f0-4835-963c-b871f3ee2a2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:52.924793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd689e-15f0-4835-963c-b871f3ee2a2f None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:52.936811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 645/1272] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:51:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:52.943093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8841281-2c76-4ed7-950e-5dbde6c90fe3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:53.032042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:53.041444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8841281-2c76-4ed7-950e-5dbde6c90fe3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:53.074546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c4c4f-5b26-453b-9e07-605fc3091a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:53.075186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c4c4f-5b26-453b-9e07-605fc3091a63 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:53.075465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c4c4f-5b26-453b-9e07-605fc3091a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:53.075738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c4c4f-5b26-453b-9e07-605fc3091a63 None admin] PATH_INFO: `/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/8f860912e1f0406cadd70da440b2ede2/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:53.077647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c4c4f-5b26-453b-9e07-605fc3091a63 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e40bb661d1a4f5cadf826e3a93af104, user_id=8f860912e1f0406cadd70da440b2ede2, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:53.079052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c4c4f-5b26-453b-9e07-605fc3091a63 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:53.091714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 628/1273] 10.222.0.78 () {58 vars in 1439 bytes} [Tue Aug 30 00:51:52 2022] PUT /identity/v3/projects/8e40bb661d1a4f5cadf826e3a93af104/users/8f860912e1f0406cadd70da440b2ede2/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:51:54.999235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49b2d-b1ef-4cfd-88ac-6b47dd67ccdc None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:54.999806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49b2d-b1ef-4cfd-88ac-6b47dd67ccdc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:54.999806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49b2d-b1ef-4cfd-88ac-6b47dd67ccdc None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:55.001232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 646/1274] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:55.006112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46fb43-f096-447c-a807-64d6b7a4d780 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:55.006528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46fb43-f096-447c-a807-64d6b7a4d780 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:55.006659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46fb43-f096-447c-a807-64d6b7a4d780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:55.025825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8b46fb43-f096-447c-a807-64d6b7a4d780 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:55.027002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8b46fb43-f096-447c-a807-64d6b7a4d780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:55.091179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 629/1275] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:55.362078 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49b2d-b1ef-4cfd-88ac-6b47dd67ccdc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:55.464347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49b2d-b1ef-4cfd-88ac-6b47dd67ccdc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:55.481117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:55.497961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ab879-e8d2-4207-9887-6aad58725d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:55.498573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ab879-e8d2-4207-9887-6aad58725d7c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:55.498845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ab879-e8d2-4207-9887-6aad58725d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:55.499110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ab879-e8d2-4207-9887-6aad58725d7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:55.509494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ab879-e8d2-4207-9887-6aad58725d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ca3ab879-e8d2-4207-9887-6aad58725d7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:55.514081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:55.515175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:55.516378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 647/1276] 10.222.0.78 () {56 vars in 1096 bytes} [Tue Aug 30 00:51:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:55.521184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46fb43-f096-447c-a807-64d6b7a4d780 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:55.598983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:55.618085 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46fb43-f096-447c-a807-64d6b7a4d780 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:55.651078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edfc18-7919-4632-a9fb-ad2a4261fa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:55.651669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edfc18-7919-4632-a9fb-ad2a4261fa57 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:55.651947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edfc18-7919-4632-a9fb-ad2a4261fa57 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:55.652216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edfc18-7919-4632-a9fb-ad2a4261fa57 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:55.652868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edfc18-7919-4632-a9fb-ad2a4261fa57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:55.653199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edfc18-7919-4632-a9fb-ad2a4261fa57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:55.654555 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edfc18-7919-4632-a9fb-ad2a4261fa57 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:55.662295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 630/1277] 10.222.0.78 () {56 vars in 1099 bytes} [Tue Aug 30 00:51:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:55.668090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ab879-e8d2-4207-9887-6aad58725d7c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:55.778008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ab879-e8d2-4207-9887-6aad58725d7c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:55.811923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0159f-ba00-428c-a82c-fdf21f503128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:55.812535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0159f-ba00-428c-a82c-fdf21f503128 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:55.812812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0159f-ba00-428c-a82c-fdf21f503128 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:55.813107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0159f-ba00-428c-a82c-fdf21f503128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:55.823097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0159f-ba00-428c-a82c-fdf21f503128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:55.824523 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0159f-ba00-428c-a82c-fdf21f503128 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e2e0159f-ba00-428c-a82c-fdf21f503128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:55.838544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:55.839692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:55.841061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:55.841061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:55.841061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:55.841061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 648/1278] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:55.845559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edfc18-7919-4632-a9fb-ad2a4261fa57 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:55.941896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edfc18-7919-4632-a9fb-ad2a4261fa57 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:55.973921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5543e3-90de-4986-8aed-25c2e254b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:55.974504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5543e3-90de-4986-8aed-25c2e254b35a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:55.974781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5543e3-90de-4986-8aed-25c2e254b35a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:55.975051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5543e3-90de-4986-8aed-25c2e254b35a None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:55.976352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5543e3-90de-4986-8aed-25c2e254b35a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:55.976649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5543e3-90de-4986-8aed-25c2e254b35a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:55.978022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5543e3-90de-4986-8aed-25c2e254b35a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:55.989504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 631/1279] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:55.996144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0159f-ba00-428c-a82c-fdf21f503128 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:56.026159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:56.096656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0159f-ba00-428c-a82c-fdf21f503128 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:56.129434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce150044-061b-4543-a497-7862e7b5111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:56.130045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce150044-061b-4543-a497-7862e7b5111c None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:56.130319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce150044-061b-4543-a497-7862e7b5111c None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:56.130585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce150044-061b-4543-a497-7862e7b5111c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:56.137957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce150044-061b-4543-a497-7862e7b5111c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:56.139377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce150044-061b-4543-a497-7862e7b5111c None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-ce150044-061b-4543-a497-7862e7b5111c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:51:56.151025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:56.152113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:56.153478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 649/1280] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:56.158027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5543e3-90de-4986-8aed-25c2e254b35a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:56.161744 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:56.257930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5543e3-90de-4986-8aed-25c2e254b35a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:56.291901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f6be9-d191-4ac2-9bf2-aaa840898b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:56.292497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f6be9-d191-4ac2-9bf2-aaa840898b37 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:56.292779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f6be9-d191-4ac2-9bf2-aaa840898b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:56.293052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f6be9-d191-4ac2-9bf2-aaa840898b37 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:56.293803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f6be9-d191-4ac2-9bf2-aaa840898b37 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:56.294109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f6be9-d191-4ac2-9bf2-aaa840898b37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:56.295535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f6be9-d191-4ac2-9bf2-aaa840898b37 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:56.312845 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 632/1281] 10.222.0.78 () {56 vars in 1143 bytes} [Tue Aug 30 00:51:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:56.318844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce150044-061b-4543-a497-7862e7b5111c None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:56.407801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:56.418244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce150044-061b-4543-a497-7862e7b5111c None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:56.451086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e457ea5-ce4e-4bd7-b62f-c83cc77cd4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:56.451682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e457ea5-ce4e-4bd7-b62f-c83cc77cd4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:56.451961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e457ea5-ce4e-4bd7-b62f-c83cc77cd4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:56.452231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e457ea5-ce4e-4bd7-b62f-c83cc77cd4b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:56.517546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e457ea5-ce4e-4bd7-b62f-c83cc77cd4b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:56.518987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e457ea5-ce4e-4bd7-b62f-c83cc77cd4b3 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:56.605062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:56.614967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 650/1282] 10.222.0.78 () {58 vars in 1282 bytes} [Tue Aug 30 00:51:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:51:56.620732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f6be9-d191-4ac2-9bf2-aaa840898b37 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:56.680039 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:56.721541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f6be9-d191-4ac2-9bf2-aaa840898b37 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:56.754831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3184b9-8abe-49fe-b489-6b68ce180a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:56.755448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3184b9-8abe-49fe-b489-6b68ce180a53 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:56.755729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3184b9-8abe-49fe-b489-6b68ce180a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:56.755998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3184b9-8abe-49fe-b489-6b68ce180a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:56.766591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3184b9-8abe-49fe-b489-6b68ce180a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:56.768020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3184b9-8abe-49fe-b489-6b68ce180a53 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-eb3184b9-8abe-49fe-b489-6b68ce180a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:56.782875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:56.783996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:51:56.784974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:56.784974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:51:56.784974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:56.784974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 633/1283] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:56.789773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e457ea5-ce4e-4bd7-b62f-c83cc77cd4b3 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:56.886677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e457ea5-ce4e-4bd7-b62f-c83cc77cd4b3 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:56.919158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe334d8f-cae3-4b6d-99c9-f876cc2b53b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:56.919749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe334d8f-cae3-4b6d-99c9-f876cc2b53b2 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:56.920027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe334d8f-cae3-4b6d-99c9-f876cc2b53b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:56.920296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe334d8f-cae3-4b6d-99c9-f876cc2b53b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:56.920922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe334d8f-cae3-4b6d-99c9-f876cc2b53b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:56.921251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe334d8f-cae3-4b6d-99c9-f876cc2b53b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:56.922601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe334d8f-cae3-4b6d-99c9-f876cc2b53b2 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:56.934050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 651/1284] 10.222.0.78 () {56 vars in 1109 bytes} [Tue Aug 30 00:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:56.939802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3184b9-8abe-49fe-b489-6b68ce180a53 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:57.040396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3184b9-8abe-49fe-b489-6b68ce180a53 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:57.054142 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:57.080637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e12e28-0eae-4ec8-8cbd-7ce4abb5188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:57.081273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12e28-0eae-4ec8-8cbd-7ce4abb5188c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:57.081552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12e28-0eae-4ec8-8cbd-7ce4abb5188c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:57.081819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12e28-0eae-4ec8-8cbd-7ce4abb5188c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:57.088635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e12e28-0eae-4ec8-8cbd-7ce4abb5188c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:57.090075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e12e28-0eae-4ec8-8cbd-7ce4abb5188c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-50e12e28-0eae-4ec8-8cbd-7ce4abb5188c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:51:57.101739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:57.102776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:57.103703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:51:57.103703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:51:57.103703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:51:57.103703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:51:57.103703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:51:57.103703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:51:57.103703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:51:57.103703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Aug 30 00:51:57.103703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 634/1285] 10.222.0.78 () {56 vars in 1111 bytes} [Tue Aug 30 00:51:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:51:57.108666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe334d8f-cae3-4b6d-99c9-f876cc2b53b2 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:57.158601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:57.207384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe334d8f-cae3-4b6d-99c9-f876cc2b53b2 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:57.239801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d31d62-f9b4-4c0a-aa1a-6db7452306f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:57.240403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d31d62-f9b4-4c0a-aa1a-6db7452306f5 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:57.240683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d31d62-f9b4-4c0a-aa1a-6db7452306f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:57.241063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d31d62-f9b4-4c0a-aa1a-6db7452306f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:57.241823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d31d62-f9b4-4c0a-aa1a-6db7452306f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:57.242128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d31d62-f9b4-4c0a-aa1a-6db7452306f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:57.243554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d31d62-f9b4-4c0a-aa1a-6db7452306f5 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:57.255880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 652/1286] 10.222.0.78 () {56 vars in 1148 bytes} [Tue Aug 30 00:51:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:57.262790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e12e28-0eae-4ec8-8cbd-7ce4abb5188c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:57.360930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e12e28-0eae-4ec8-8cbd-7ce4abb5188c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:57.393726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fc174-83a9-4cca-9c50-95f410c461ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:57.394312 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fc174-83a9-4cca-9c50-95f410c461ea None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:57.394591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fc174-83a9-4cca-9c50-95f410c461ea None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:57.394863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fc174-83a9-4cca-9c50-95f410c461ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:57.395577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fc174-83a9-4cca-9c50-95f410c461ea None admin] RBAC: Adding query filter params (role.id=7e63743509de412fbbf1c7b60a8ae2ce, scope.project.id=8e40bb661d1a4f5cadf826e3a93af104, user.id=8f860912e1f0406cadd70da440b2ede2) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:57.395885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fc174-83a9-4cca-9c50-95f410c461ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:57.398149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fc174-83a9-4cca-9c50-95f410c461ea None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:57.403488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 635/1287] 10.222.0.78 () {56 vars in 1374 bytes} [Tue Aug 30 00:51:57 2022] GET /identity/v3/role_assignments?user.id=8f860912e1f0406cadd70da440b2ede2&scope.project.id=8e40bb661d1a4f5cadf826e3a93af104&role.id=7e63743509de412fbbf1c7b60a8ae2ce => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:59.323982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6d3d5-c0b9-4e08-a24b-4e541c8dac17 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:59.324511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6d3d5-c0b9-4e08-a24b-4e541c8dac17 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:59.324511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6d3d5-c0b9-4e08-a24b-4e541c8dac17 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:59.325879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 653/1288] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:59.330839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad878a72-c26d-4595-bd15-3ee031364295 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:59.331270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad878a72-c26d-4595-bd15-3ee031364295 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:59.331366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad878a72-c26d-4595-bd15-3ee031364295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:59.349630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ad878a72-c26d-4595-bd15-3ee031364295 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:59.351336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ad878a72-c26d-4595-bd15-3ee031364295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:59.380421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:59.418595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 636/1289] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:59.691660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6d3d5-c0b9-4e08-a24b-4e541c8dac17 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:59.806121 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6d3d5-c0b9-4e08-a24b-4e541c8dac17 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:59.816513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:59.840571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407435e5-b140-4400-b0f6-ce8aa2770412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:59.841208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407435e5-b140-4400-b0f6-ce8aa2770412 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:59.841484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407435e5-b140-4400-b0f6-ce8aa2770412 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:59.841747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407435e5-b140-4400-b0f6-ce8aa2770412 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:59.842307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407435e5-b140-4400-b0f6-ce8aa2770412 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:59.843676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407435e5-b140-4400-b0f6-ce8aa2770412 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-407435e5-b140-4400-b0f6-ce8aa2770412 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:59.852440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:59.853651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:51:59.854509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 654/1290] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:51:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:59.859511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad878a72-c26d-4595-bd15-3ee031364295 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:59.956471 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad878a72-c26d-4595-bd15-3ee031364295 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:59.989051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e318c9-64a8-44cf-ad92-b9c0bd092ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:59.989657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e318c9-64a8-44cf-ad92-b9c0bd092ecb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:59.989934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e318c9-64a8-44cf-ad92-b9c0bd092ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:59.990196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e318c9-64a8-44cf-ad92-b9c0bd092ecb None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:59.991343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e318c9-64a8-44cf-ad92-b9c0bd092ecb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:59.991641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e318c9-64a8-44cf-ad92-b9c0bd092ecb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:59.992976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e318c9-64a8-44cf-ad92-b9c0bd092ecb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:59.997627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 637/1291] 10.222.0.78 () {56 vars in 1116 bytes} [Tue Aug 30 00:51:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:00.003433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407435e5-b140-4400-b0f6-ce8aa2770412 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:00.099805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407435e5-b140-4400-b0f6-ce8aa2770412 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:00.132338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df4ff8-db78-4f3f-80ec-6ab651e0a4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:00.133003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df4ff8-db78-4f3f-80ec-6ab651e0a4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:00.133305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df4ff8-db78-4f3f-80ec-6ab651e0a4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:00.133569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df4ff8-db78-4f3f-80ec-6ab651e0a4e1 None admin] PATH_INFO: `/v3/services` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:00.134177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df4ff8-db78-4f3f-80ec-6ab651e0a4e1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:00.134471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df4ff8-db78-4f3f-80ec-6ab651e0a4e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:00.135805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df4ff8-db78-4f3f-80ec-6ab651e0a4e1 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:00.140029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 655/1292] 10.222.0.78 () {56 vars in 1116 bytes} [Tue Aug 30 00:52:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:02.046677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3303474-ea59-45c4-9a73-c53aaa4f40f1 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:02.047216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3303474-ea59-45c4-9a73-c53aaa4f40f1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:02.047330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3303474-ea59-45c4-9a73-c53aaa4f40f1 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:02.048583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 638/1293] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:52:02.054033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be87a46-f88d-48ba-b639-faa5330d5324 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:02.054470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be87a46-f88d-48ba-b639-faa5330d5324 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:02.054607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be87a46-f88d-48ba-b639-faa5330d5324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:02.073220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8be87a46-f88d-48ba-b639-faa5330d5324 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:02.074292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8be87a46-f88d-48ba-b639-faa5330d5324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:02.102692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:02.139611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 656/1294] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:52:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:52:02.410737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3303474-ea59-45c4-9a73-c53aaa4f40f1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:02.441821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:02.511936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3303474-ea59-45c4-9a73-c53aaa4f40f1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:02.544974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60dd14-3bd6-4f2e-908d-6c9f9b3dde08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:02.545589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60dd14-3bd6-4f2e-908d-6c9f9b3dde08 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:02.545868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60dd14-3bd6-4f2e-908d-6c9f9b3dde08 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:02.546137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60dd14-3bd6-4f2e-908d-6c9f9b3dde08 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:02.546784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60dd14-3bd6-4f2e-908d-6c9f9b3dde08 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:02.548117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60dd14-3bd6-4f2e-908d-6c9f9b3dde08 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:02.562289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 639/1295] 10.222.0.78 () {60 vars in 1141 bytes} [Tue Aug 30 00:52:02 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:52:04.474437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b833699-43f7-4144-adf6-c09f3965b891 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:04.474973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b833699-43f7-4144-adf6-c09f3965b891 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:04.475094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b833699-43f7-4144-adf6-c09f3965b891 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:04.476885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 657/1296] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:52:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:52:04.482065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00044b-d06c-400a-9734-e6c3d49e77b1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:04.482453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00044b-d06c-400a-9734-e6c3d49e77b1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:04.482617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00044b-d06c-400a-9734-e6c3d49e77b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:04.500363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0a00044b-d06c-400a-9734-e6c3d49e77b1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:04.501573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0a00044b-d06c-400a-9734-e6c3d49e77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:04.572370 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 640/1297] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:52:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:52:04.854748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b833699-43f7-4144-adf6-c09f3965b891 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:04.960629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b833699-43f7-4144-adf6-c09f3965b891 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:04.992460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df88b0-83f2-4272-a858-547e30dee4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:04.993066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df88b0-83f2-4272-a858-547e30dee4dd None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:04.993364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df88b0-83f2-4272-a858-547e30dee4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:04.993630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df88b0-83f2-4272-a858-547e30dee4dd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:04.994201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df88b0-83f2-4272-a858-547e30dee4dd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:04.995540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df88b0-83f2-4272-a858-547e30dee4dd None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-d4df88b0-83f2-4272-a858-547e30dee4dd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:05.003924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:05.005036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:52:05.006062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 658/1298] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:52:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:05.011529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00044b-d06c-400a-9734-e6c3d49e77b1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:05.029614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:05.110624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00044b-d06c-400a-9734-e6c3d49e77b1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:05.143592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6d5-322d-4c78-b96e-5363d7f49cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:05.144183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6d5-322d-4c78-b96e-5363d7f49cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:05.144463 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6d5-322d-4c78-b96e-5363d7f49cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:05.144737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6d5-322d-4c78-b96e-5363d7f49cc1 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:05.145386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dc6d5-322d-4c78-b96e-5363d7f49cc1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:05.145726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dc6d5-322d-4c78-b96e-5363d7f49cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:05.147114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dc6d5-322d-4c78-b96e-5363d7f49cc1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:05.152090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 641/1299] 10.222.0.78 () {56 vars in 1116 bytes} [Tue Aug 30 00:52:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:52:05.158001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df88b0-83f2-4272-a858-547e30dee4dd None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:05.189487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:05.257154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df88b0-83f2-4272-a858-547e30dee4dd None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:05.290235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955f94d-6e4e-40bc-a6c1-6c620748e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:05.290835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955f94d-6e4e-40bc-a6c1-6c620748e307 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:05.291110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955f94d-6e4e-40bc-a6c1-6c620748e307 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:05.291377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955f94d-6e4e-40bc-a6c1-6c620748e307 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:05.292095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955f94d-6e4e-40bc-a6c1-6c620748e307 None admin] RBAC: Adding query filter params (interface=public, service_id=7402af23be6a4fd59f266f6c82961600, region_id=RegionOne) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:05.292398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955f94d-6e4e-40bc-a6c1-6c620748e307 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:05.293796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955f94d-6e4e-40bc-a6c1-6c620748e307 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:05.299326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 659/1300] 10.222.0.78 () {56 vars in 1251 bytes} [Tue Aug 30 00:52:05 2022] GET /identity/v3/endpoints?service_id=7402af23be6a4fd59f266f6c82961600&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:05.305844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6d5-322d-4c78-b96e-5363d7f49cc1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:05.386335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:05.404716 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6d5-322d-4c78-b96e-5363d7f49cc1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:05.439159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d85e63-a23e-486a-b148-b99fcb55f032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:05.439753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d85e63-a23e-486a-b148-b99fcb55f032 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:05.440026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d85e63-a23e-486a-b148-b99fcb55f032 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:05.440291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d85e63-a23e-486a-b148-b99fcb55f032 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:05.440882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d85e63-a23e-486a-b148-b99fcb55f032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:05.442252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d85e63-a23e-486a-b148-b99fcb55f032 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:05.449494 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 642/1301] 10.222.0.78 () {56 vars in 1087 bytes} [Tue Aug 30 00:52:05 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:52:07.348105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303618ff-e51c-4e32-8b19-1a47ca70fe4a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:07.348654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303618ff-e51c-4e32-8b19-1a47ca70fe4a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:07.348654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303618ff-e51c-4e32-8b19-1a47ca70fe4a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:07.350202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 660/1302] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:52:07.355181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025002-d445-46b4-a60a-ad65bd79c617 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:07.355554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025002-d445-46b4-a60a-ad65bd79c617 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:07.355718 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025002-d445-46b4-a60a-ad65bd79c617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:07.374293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-37025002-d445-46b4-a60a-ad65bd79c617 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:07.375453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-37025002-d445-46b4-a60a-ad65bd79c617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:07.439524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 643/1303] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:52:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:52:07.721262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303618ff-e51c-4e32-8b19-1a47ca70fe4a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:07.812460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:07.820135 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303618ff-e51c-4e32-8b19-1a47ca70fe4a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:07.852382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8e81c-192d-4225-9554-926841911f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:07.852979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8e81c-192d-4225-9554-926841911f48 None admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:07.853281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8e81c-192d-4225-9554-926841911f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:07.853549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8e81c-192d-4225-9554-926841911f48 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:07.854111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8e81c-192d-4225-9554-926841911f48 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:07.855452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8e81c-192d-4225-9554-926841911f48 None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application [None req-e0a8e81c-192d-4225-9554-926841911f48 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:07.864104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:07.865279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: ERROR keystone.server.flask.application Aug 30 00:52:07.866449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 661/1304] 10.222.0.78 () {56 vars in 1117 bytes} [Tue Aug 30 00:52:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:07.871360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37025002-d445-46b4-a60a-ad65bd79c617 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:07.944948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:07.967788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37025002-d445-46b4-a60a-ad65bd79c617 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:07.999467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8852d67-4122-4e4e-930a-06075fc64b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:07.999991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8852d67-4122-4e4e-930a-06075fc64b26 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:08.000265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8852d67-4122-4e4e-930a-06075fc64b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:08.000531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8852d67-4122-4e4e-930a-06075fc64b26 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:08.001274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8852d67-4122-4e4e-930a-06075fc64b26 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:08.001570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8852d67-4122-4e4e-930a-06075fc64b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:08.002905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8852d67-4122-4e4e-930a-06075fc64b26 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:08.008348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 644/1305] 10.222.0.78 () {56 vars in 1116 bytes} [Tue Aug 30 00:52:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:08.014074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8e81c-192d-4225-9554-926841911f48 None admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:08.110504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8e81c-192d-4225-9554-926841911f48 None admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:08.143281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc228d7a-a725-4fae-abee-a2414e72dafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:08.143864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc228d7a-a725-4fae-abee-a2414e72dafb None admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:08.144140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc228d7a-a725-4fae-abee-a2414e72dafb None admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:08.144409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc228d7a-a725-4fae-abee-a2414e72dafb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:08.145069 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc228d7a-a725-4fae-abee-a2414e72dafb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:08.146450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc228d7a-a725-4fae-abee-a2414e72dafb None admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:08.165696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 662/1306] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:52:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:52:18.004915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=30 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:18.161174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:12.067195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4719892-7625-46ed-984d-7bacc591f4e3 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:12.068226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4719892-7625-46ed-984d-7bacc591f4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:12.068733 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4719892-7625-46ed-984d-7bacc591f4e3 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:12.074842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 645/1307] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:12 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:12.077258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71ee69-d5e3-4589-b056-b180d4da1d45 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:12.077704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71ee69-d5e3-4589-b056-b180d4da1d45 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:12.078173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71ee69-d5e3-4589-b056-b180d4da1d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:12.168313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-af71ee69-d5e3-4589-b056-b180d4da1d45 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:12.169541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-af71ee69-d5e3-4589-b056-b180d4da1d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:12.289099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 663/1308] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:12.624248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90945d6-c361-4b27-a013-e48e35476b19 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:12.624534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90945d6-c361-4b27-a013-e48e35476b19 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:12.625016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90945d6-c361-4b27-a013-e48e35476b19 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:12.626412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 646/1309] 10.222.0.78 () {54 vars in 965 bytes} [Tue Aug 30 00:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:12.632645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7851719-9034-45e8-ad19-e9e9a4fedf4d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:12.632937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7851719-9034-45e8-ad19-e9e9a4fedf4d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:12.633257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7851719-9034-45e8-ad19-e9e9a4fedf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:12.718143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d7851719-9034-45e8-ad19-e9e9a4fedf4d None None] MFA Rules not processed for user `1a5d841380c243209f55314b545ff56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:12.719361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d7851719-9034-45e8-ad19-e9e9a4fedf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:12.741986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:12.803612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 664/1310] 10.222.0.78 () {58 vars in 1062 bytes} [Tue Aug 30 00:53:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:12.813016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90945d6-c361-4b27-a013-e48e35476b19 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:12.814959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c90945d6-c361-4b27-a013-e48e35476b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:12.945449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90945d6-c361-4b27-a013-e48e35476b19 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:13.030029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9da6f-89a3-46cc-a918-5ba0b5c6574f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:13.030644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9da6f-89a3-46cc-a918-5ba0b5c6574f service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:13.030919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9da6f-89a3-46cc-a918-5ba0b5c6574f service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:13.031184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9da6f-89a3-46cc-a918-5ba0b5c6574f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:13.032886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-62d9da6f-89a3-46cc-a918-5ba0b5c6574f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:13.097718 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:13.158099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9da6f-89a3-46cc-a918-5ba0b5c6574f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:13.160323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9da6f-89a3-46cc-a918-5ba0b5c6574f service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:13.180466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 647/1311] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:15.422601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e6498-00ef-4077-8550-ac497df30201 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:15.423155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e6498-00ef-4077-8550-ac497df30201 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:15.423155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e6498-00ef-4077-8550-ac497df30201 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:15.424804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 665/1312] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:15.430038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9357ac7-5d1c-45e5-ace0-eeef156bce73 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:15.430458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9357ac7-5d1c-45e5-ace0-eeef156bce73 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:15.430633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9357ac7-5d1c-45e5-ace0-eeef156bce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:15.526673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f9357ac7-5d1c-45e5-ace0-eeef156bce73 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:15.527838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f9357ac7-5d1c-45e5-ace0-eeef156bce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:15.639861 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 648/1313] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:15.928599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e6498-00ef-4077-8550-ac497df30201 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:15.930331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-406e6498-00ef-4077-8550-ac497df30201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:16.040054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e6498-00ef-4077-8550-ac497df30201 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:16.110424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:16.114003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185e9fd-1155-46d0-b6fa-13c0d3f29919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:16.114618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185e9fd-1155-46d0-b6fa-13c0d3f29919 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:16.115103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185e9fd-1155-46d0-b6fa-13c0d3f29919 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:16.115424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185e9fd-1155-46d0-b6fa-13c0d3f29919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:16.117279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1185e9fd-1155-46d0-b6fa-13c0d3f29919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:16.226678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185e9fd-1155-46d0-b6fa-13c0d3f29919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:16.228122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185e9fd-1155-46d0-b6fa-13c0d3f29919 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:16.247133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 666/1314] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:18.468962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbab1ba-e703-462d-a15b-3b09c3571540 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:18.469550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbab1ba-e703-462d-a15b-3b09c3571540 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:18.469550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbab1ba-e703-462d-a15b-3b09c3571540 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:18.471828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 649/1315] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:18.477027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13333b2f-9bc5-4278-a1ea-a8634bc7692e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:18.477480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13333b2f-9bc5-4278-a1ea-a8634bc7692e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:18.477695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13333b2f-9bc5-4278-a1ea-a8634bc7692e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:18.544080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-13333b2f-9bc5-4278-a1ea-a8634bc7692e None None] MFA Rules not processed for user `42f1327f05514bbeb23ec0f553202fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:18.545293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-13333b2f-9bc5-4278-a1ea-a8634bc7692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:18.598624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:18.644009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 667/1316] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:18.942097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbab1ba-e703-462d-a15b-3b09c3571540 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:18.943648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9bbab1ba-e703-462d-a15b-3b09c3571540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:18.953309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:19.044279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbab1ba-e703-462d-a15b-3b09c3571540 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:19.117820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4677baf-1229-4831-be21-1ae3d411ddcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:19.118498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4677baf-1229-4831-be21-1ae3d411ddcf service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:19.118847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4677baf-1229-4831-be21-1ae3d411ddcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:19.119212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4677baf-1229-4831-be21-1ae3d411ddcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:19.121512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d4677baf-1229-4831-be21-1ae3d411ddcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:19.183472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:19.232396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4677baf-1229-4831-be21-1ae3d411ddcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:19.233811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4677baf-1229-4831-be21-1ae3d411ddcf service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:19.253200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 650/1317] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:21.740830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80959a3-63db-4569-93ce-6eabf8ac2d79 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:21.741404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80959a3-63db-4569-93ce-6eabf8ac2d79 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:21.741404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80959a3-63db-4569-93ce-6eabf8ac2d79 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:21.742755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 668/1318] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:21.747791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630ab30-920f-49d3-8525-aca8ee3b91b9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:21.748326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630ab30-920f-49d3-8525-aca8ee3b91b9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:21.748326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630ab30-920f-49d3-8525-aca8ee3b91b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:21.814552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9630ab30-920f-49d3-8525-aca8ee3b91b9 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:21.815857 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9630ab30-920f-49d3-8525-aca8ee3b91b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:21.910001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 651/1319] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:22.207397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80959a3-63db-4569-93ce-6eabf8ac2d79 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:22.208949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a80959a3-63db-4569-93ce-6eabf8ac2d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:22.314334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80959a3-63db-4569-93ce-6eabf8ac2d79 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:22.384916 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f202fa1-cbd1-46a2-a5ff-841045ce2dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:22.385536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f202fa1-cbd1-46a2-a5ff-841045ce2dcf service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:22.385812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f202fa1-cbd1-46a2-a5ff-841045ce2dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:22.386079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f202fa1-cbd1-46a2-a5ff-841045ce2dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:22.387653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1f202fa1-cbd1-46a2-a5ff-841045ce2dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:22.395567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:22.497132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f202fa1-cbd1-46a2-a5ff-841045ce2dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:22.498501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f202fa1-cbd1-46a2-a5ff-841045ce2dcf service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:22.516709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 669/1320] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:25.099655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f24b73-b04e-4d04-b8a1-25574dbd89fb None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:25.100249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f24b73-b04e-4d04-b8a1-25574dbd89fb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:25.100249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f24b73-b04e-4d04-b8a1-25574dbd89fb None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:25.101588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 652/1321] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:25.106689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22def4a-dda1-4e34-8ef2-b1127564d587 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:25.107127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22def4a-dda1-4e34-8ef2-b1127564d587 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:25.107265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22def4a-dda1-4e34-8ef2-b1127564d587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:25.175206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d22def4a-dda1-4e34-8ef2-b1127564d587 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:25.176404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d22def4a-dda1-4e34-8ef2-b1127564d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:25.234496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:25.281326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 670/1322] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:25.584608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11902fd6-a418-48fc-a277-cb23e8313660 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:25.584892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11902fd6-a418-48fc-a277-cb23e8313660 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:25.585213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11902fd6-a418-48fc-a277-cb23e8313660 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:25.586571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 653/1323] 10.222.0.78 () {54 vars in 965 bytes} [Tue Aug 30 00:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:25.592792 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109437bc-6e60-41c1-b863-346c924b10d6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:25.593178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109437bc-6e60-41c1-b863-346c924b10d6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:25.593519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109437bc-6e60-41c1-b863-346c924b10d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:25.668118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-109437bc-6e60-41c1-b863-346c924b10d6 None None] MFA Rules not processed for user `1a5d841380c243209f55314b545ff56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:25.669361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-109437bc-6e60-41c1-b863-346c924b10d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:25.748147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 671/1324] 10.222.0.78 () {58 vars in 1062 bytes} [Tue Aug 30 00:53:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:25.757870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11902fd6-a418-48fc-a277-cb23e8313660 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:25.759342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-11902fd6-a418-48fc-a277-cb23e8313660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:25.778575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:25.866637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11902fd6-a418-48fc-a277-cb23e8313660 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:25.944585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefeec0b-94e7-4a47-8934-4dce054d7fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:25.945226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeec0b-94e7-4a47-8934-4dce054d7fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:25.945504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeec0b-94e7-4a47-8934-4dce054d7fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:25.945770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeec0b-94e7-4a47-8934-4dce054d7fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:25.947381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cefeec0b-94e7-4a47-8934-4dce054d7fa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:26.018747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:26.057896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefeec0b-94e7-4a47-8934-4dce054d7fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:26.059256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefeec0b-94e7-4a47-8934-4dce054d7fa8 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:26.078108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 654/1325] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:28.283654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d155ce-58bf-4663-b048-8cbef463329c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:28.284226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d155ce-58bf-4663-b048-8cbef463329c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:28.284226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d155ce-58bf-4663-b048-8cbef463329c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:28.285584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 672/1326] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:28.290886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2d8b9-39ed-4293-95fc-6bc43709281c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:28.291345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2d8b9-39ed-4293-95fc-6bc43709281c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:28.291468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2d8b9-39ed-4293-95fc-6bc43709281c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:28.359008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6cb2d8b9-39ed-4293-95fc-6bc43709281c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:28.360208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6cb2d8b9-39ed-4293-95fc-6bc43709281c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:28.470158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 655/1327] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:28.758648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d155ce-58bf-4663-b048-8cbef463329c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:28.760176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-06d155ce-58bf-4663-b048-8cbef463329c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:28.821840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:28.867227 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d155ce-58bf-4663-b048-8cbef463329c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:28.945142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bebe0-a225-47ff-becd-8423d4ff3687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:28.945749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bebe0-a225-47ff-becd-8423d4ff3687 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:28.946027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bebe0-a225-47ff-becd-8423d4ff3687 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:28.946297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bebe0-a225-47ff-becd-8423d4ff3687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:28.947906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c80bebe0-a225-47ff-becd-8423d4ff3687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:29.059063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:29.061966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bebe0-a225-47ff-becd-8423d4ff3687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:29.064377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bebe0-a225-47ff-becd-8423d4ff3687 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:29.084054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 673/1328] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:31.096755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe7e89-7970-4e90-a5dc-ec3333ab5e0b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:31.097302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe7e89-7970-4e90-a5dc-ec3333ab5e0b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:31.097491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe7e89-7970-4e90-a5dc-ec3333ab5e0b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:31.098851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 656/1329] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:31.104564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8ce88-4a1d-44fd-a6ae-c1260c367ae8 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:31.105059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8ce88-4a1d-44fd-a6ae-c1260c367ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:31.105252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8ce88-4a1d-44fd-a6ae-c1260c367ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:31.176159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-48e8ce88-4a1d-44fd-a6ae-c1260c367ae8 None None] MFA Rules not processed for user `42f1327f05514bbeb23ec0f553202fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:31.177424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-48e8ce88-4a1d-44fd-a6ae-c1260c367ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:31.273166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 674/1330] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:31.561646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe7e89-7970-4e90-a5dc-ec3333ab5e0b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:31.563185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-20fe7e89-7970-4e90-a5dc-ec3333ab5e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:31.594458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:31.669307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe7e89-7970-4e90-a5dc-ec3333ab5e0b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:31.748976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c42dcd-adcd-4857-9937-06c5e248e3e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:31.749625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c42dcd-adcd-4857-9937-06c5e248e3e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:31.749929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c42dcd-adcd-4857-9937-06c5e248e3e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:31.750187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c42dcd-adcd-4857-9937-06c5e248e3e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:31.752844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a7c42dcd-adcd-4857-9937-06c5e248e3e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:31.835854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:31.870119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c42dcd-adcd-4857-9937-06c5e248e3e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:31.871556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c42dcd-adcd-4857-9937-06c5e248e3e8 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:31.897285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 657/1331] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:34.247066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0367f5-dd29-4285-9369-a21505169ad2 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:34.247560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0367f5-dd29-4285-9369-a21505169ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:34.247895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0367f5-dd29-4285-9369-a21505169ad2 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:34.249655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 675/1332] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:34.255483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce8911-1927-428a-867e-ddd288854102 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:34.255878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce8911-1927-428a-867e-ddd288854102 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:34.256034 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce8911-1927-428a-867e-ddd288854102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:34.322290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3fce8911-1927-428a-867e-ddd288854102 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:34.323512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3fce8911-1927-428a-867e-ddd288854102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:34.427632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 658/1333] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:34.711183 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0367f5-dd29-4285-9369-a21505169ad2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:34.712772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8c0367f5-dd29-4285-9369-a21505169ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:34.781304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:34.816653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0367f5-dd29-4285-9369-a21505169ad2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:34.890465 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9c42f-776d-48a2-837e-9f4abe993a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:34.891056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9c42f-776d-48a2-837e-9f4abe993a2f service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:34.891358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9c42f-776d-48a2-837e-9f4abe993a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:34.891635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9c42f-776d-48a2-837e-9f4abe993a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:34.893219 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-78f9c42f-776d-48a2-837e-9f4abe993a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:35.000757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9c42f-776d-48a2-837e-9f4abe993a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:35.002190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9c42f-776d-48a2-837e-9f4abe993a2f service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:35.012003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:35.022058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 676/1334] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:36.982965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb625ab-24a3-4346-935c-6309ecddf029 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:36.983531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb625ab-24a3-4346-935c-6309ecddf029 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:36.983627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb625ab-24a3-4346-935c-6309ecddf029 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:36.984976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 659/1335] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:36.990518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4892e0-5c07-418e-ab99-756c6bca5808 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:36.990954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4892e0-5c07-418e-ab99-756c6bca5808 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:36.991051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4892e0-5c07-418e-ab99-756c6bca5808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:37.055604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1a4892e0-5c07-418e-ab99-756c6bca5808 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:37.056799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1a4892e0-5c07-418e-ab99-756c6bca5808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:37.150846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 677/1336] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:37.435691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb625ab-24a3-4346-935c-6309ecddf029 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:37.437257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ebb625ab-24a3-4346-935c-6309ecddf029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:37.478604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:37.543119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb625ab-24a3-4346-935c-6309ecddf029 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:37.615174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646b1f6-5b71-4274-b1e9-29c6d7e14495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:37.615771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646b1f6-5b71-4274-b1e9-29c6d7e14495 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:37.616042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646b1f6-5b71-4274-b1e9-29c6d7e14495 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:37.616305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646b1f6-5b71-4274-b1e9-29c6d7e14495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:37.617906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e646b1f6-5b71-4274-b1e9-29c6d7e14495 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:37.700887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:37.727022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646b1f6-5b71-4274-b1e9-29c6d7e14495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:37.728427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646b1f6-5b71-4274-b1e9-29c6d7e14495 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:37.745755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 660/1337] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:40.108589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5afc2-4598-44a3-a1f3-3df408284c77 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:40.109154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5afc2-4598-44a3-a1f3-3df408284c77 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:40.109251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5afc2-4598-44a3-a1f3-3df408284c77 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:40.110695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 678/1338] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:40.116108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80566b-edb9-4279-8d9c-54f9ad420162 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:40.116558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80566b-edb9-4279-8d9c-54f9ad420162 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:40.116667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80566b-edb9-4279-8d9c-54f9ad420162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:40.182043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ed80566b-edb9-4279-8d9c-54f9ad420162 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:40.183175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ed80566b-edb9-4279-8d9c-54f9ad420162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:40.279240 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 661/1339] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:53:40.567024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5afc2-4598-44a3-a1f3-3df408284c77 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:40.568565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-dbf5afc2-4598-44a3-a1f3-3df408284c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:40.645678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:40.673324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5afc2-4598-44a3-a1f3-3df408284c77 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:40.744878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a67c18-9ad9-438f-8211-aa7d6bbe5f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:40.745510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a67c18-9ad9-438f-8211-aa7d6bbe5f99 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:40.745778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a67c18-9ad9-438f-8211-aa7d6bbe5f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:40.746047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a67c18-9ad9-438f-8211-aa7d6bbe5f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:40.747612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-06a67c18-9ad9-438f-8211-aa7d6bbe5f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:40.859077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a67c18-9ad9-438f-8211-aa7d6bbe5f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:40.860979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a67c18-9ad9-438f-8211-aa7d6bbe5f99 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:40.879444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 679/1340] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:44.854921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd3a5f-8388-48a2-a7a1-95b6b38e0b0c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:44.855458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd3a5f-8388-48a2-a7a1-95b6b38e0b0c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:44.855458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd3a5f-8388-48a2-a7a1-95b6b38e0b0c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:44.856820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 662/1341] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:44.862242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431c173-48aa-428b-b083-de89ff24efa4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:44.862652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431c173-48aa-428b-b083-de89ff24efa4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:44.862901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431c173-48aa-428b-b083-de89ff24efa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:44.868575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:44.933981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0431c173-48aa-428b-b083-de89ff24efa4 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:44.935207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0431c173-48aa-428b-b083-de89ff24efa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:45.036791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 680/1342] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:45.322970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd3a5f-8388-48a2-a7a1-95b6b38e0b0c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:45.324603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-59cd3a5f-8388-48a2-a7a1-95b6b38e0b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:45.373506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:45.429719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd3a5f-8388-48a2-a7a1-95b6b38e0b0c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:45.502849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a59ebb-27e4-4a43-b983-1ce374ff2e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:45.503447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a59ebb-27e4-4a43-b983-1ce374ff2e4f service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:45.503721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a59ebb-27e4-4a43-b983-1ce374ff2e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:45.503989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a59ebb-27e4-4a43-b983-1ce374ff2e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:45.505603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-95a59ebb-27e4-4a43-b983-1ce374ff2e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:45.597139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:45.613933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a59ebb-27e4-4a43-b983-1ce374ff2e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:45.615364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a59ebb-27e4-4a43-b983-1ce374ff2e4f service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:45.633775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 663/1343] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:48.176651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414d74d-a19b-4e1f-8106-245dbe582a5d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:48.177206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414d74d-a19b-4e1f-8106-245dbe582a5d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:48.177206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414d74d-a19b-4e1f-8106-245dbe582a5d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:48.178475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 681/1344] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:48.183904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafffed-209c-490d-85f2-c03a80e4aafa None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:48.184319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafffed-209c-490d-85f2-c03a80e4aafa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:48.184438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafffed-209c-490d-85f2-c03a80e4aafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:48.256151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-faafffed-209c-490d-85f2-c03a80e4aafa None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:48.257385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-faafffed-209c-490d-85f2-c03a80e4aafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:48.357703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 664/1345] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:48.646168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414d74d-a19b-4e1f-8106-245dbe582a5d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:48.647746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6414d74d-a19b-4e1f-8106-245dbe582a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:48.746252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:48.764101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414d74d-a19b-4e1f-8106-245dbe582a5d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:48.844153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4246f4-0d69-4deb-bf3a-798d1293d316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:48.845056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4246f4-0d69-4deb-bf3a-798d1293d316 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:48.845369 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4246f4-0d69-4deb-bf3a-798d1293d316 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:48.845640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4246f4-0d69-4deb-bf3a-798d1293d316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:48.847288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4b4246f4-0d69-4deb-bf3a-798d1293d316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:48.957387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4246f4-0d69-4deb-bf3a-798d1293d316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:48.958740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4246f4-0d69-4deb-bf3a-798d1293d316 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:48.978222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 682/1346] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:53:53.184289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b55a5-2e83-44ab-be03-750546eba6ca None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:53.184811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b55a5-2e83-44ab-be03-750546eba6ca None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:53.184890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b55a5-2e83-44ab-be03-750546eba6ca None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:53.186306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 665/1347] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:53.192194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716c880-76c1-48d0-9c7d-686734daf306 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:53.192724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716c880-76c1-48d0-9c7d-686734daf306 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:53.192868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716c880-76c1-48d0-9c7d-686734daf306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:53.209064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:53.268127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e716c880-76c1-48d0-9c7d-686734daf306 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:53.269402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e716c880-76c1-48d0-9c7d-686734daf306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:53.372070 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 683/1348] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:53.659399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b55a5-2e83-44ab-be03-750546eba6ca None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:53.660924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c53b55a5-2e83-44ab-be03-750546eba6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:53.720928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:53.767840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b55a5-2e83-44ab-be03-750546eba6ca None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:53.844657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c9892-6936-4073-a4c2-ed61ba15ba03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:53.845210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c9892-6936-4073-a4c2-ed61ba15ba03 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:53.845493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c9892-6936-4073-a4c2-ed61ba15ba03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:53.845767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c9892-6936-4073-a4c2-ed61ba15ba03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:53.847367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9a4c9892-6936-4073-a4c2-ed61ba15ba03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:53.967552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:53.972603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c9892-6936-4073-a4c2-ed61ba15ba03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:53.974084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c9892-6936-4073-a4c2-ed61ba15ba03 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:53.992678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 666/1349] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:58.952494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4c83a-0553-4620-923d-443bc5d2216b None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:58.953048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4c83a-0553-4620-923d-443bc5d2216b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:58.953202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4c83a-0553-4620-923d-443bc5d2216b None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:58.956511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 684/1350] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 00:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:58.962743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d106d4-b042-45fd-b9d5-32c6282c6746 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:58.963189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d106d4-b042-45fd-b9d5-32c6282c6746 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:58.963456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d106d4-b042-45fd-b9d5-32c6282c6746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:59.110176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e3d106d4-b042-45fd-b9d5-32c6282c6746 None None] MFA Rules not processed for user `8f860912e1f0406cadd70da440b2ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:59.111470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e3d106d4-b042-45fd-b9d5-32c6282c6746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:59.399467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 667/1351] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 00:53:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:03.370209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=59 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:03.568914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c7750-eaf4-4e99-a4e5-c6da28972a79 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:03.569352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c7750-eaf4-4e99-a4e5-c6da28972a79 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:03.569761 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c7750-eaf4-4e99-a4e5-c6da28972a79 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:03.571175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 685/1352] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 00:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:03.576656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45495475-ec8e-45f9-9765-b671e9e512be None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:03.577796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45495475-ec8e-45f9-9765-b671e9e512be None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:03.577877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45495475-ec8e-45f9-9765-b671e9e512be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:03.638090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:03.663608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-45495475-ec8e-45f9-9765-b671e9e512be None None] MFA Rules not processed for user `8f860912e1f0406cadd70da440b2ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:03.664844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-45495475-ec8e-45f9-9765-b671e9e512be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:03.778470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 668/1353] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 00:54:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:05.118950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3cbd6-1860-4a04-abc1-26bf5d894a8c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:05.119494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3cbd6-1860-4a04-abc1-26bf5d894a8c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:05.119494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3cbd6-1860-4a04-abc1-26bf5d894a8c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:05.120844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 686/1354] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 00:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:05.126697 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d2309-bf20-408d-b651-7924d1b0cc7c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:05.127105 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d2309-bf20-408d-b651-7924d1b0cc7c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:05.127244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d2309-bf20-408d-b651-7924d1b0cc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:05.204686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-474d2309-bf20-408d-b651-7924d1b0cc7c None None] MFA Rules not processed for user `8f860912e1f0406cadd70da440b2ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:05.205990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-474d2309-bf20-408d-b651-7924d1b0cc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:05.252536 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:05.328233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 669/1355] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 00:54:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:08.760680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1a3ab-2df1-47e2-84f5-72ee91f4f899 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:08.761171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1a3ab-2df1-47e2-84f5-72ee91f4f899 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:08.761335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1a3ab-2df1-47e2-84f5-72ee91f4f899 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:08.762870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 687/1356] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:08.770715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ff336-5a8b-48cd-94fc-b36db09096ec None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:08.771162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ff336-5a8b-48cd-94fc-b36db09096ec None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:08.771256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ff336-5a8b-48cd-94fc-b36db09096ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:08.870977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-858ff336-5a8b-48cd-94fc-b36db09096ec None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:08.872194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-858ff336-5a8b-48cd-94fc-b36db09096ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:09.005636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 670/1357] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:54:09.315205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085b9d-c7ff-4d07-a448-6f48d80f45ef None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:09.315578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085b9d-c7ff-4d07-a448-6f48d80f45ef None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:09.315751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085b9d-c7ff-4d07-a448-6f48d80f45ef None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:09.317303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 688/1358] 10.222.0.78 () {54 vars in 962 bytes} [Tue Aug 30 00:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:09.332375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb34bb-762f-4166-b3e7-c654d16be3c5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:09.332790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb34bb-762f-4166-b3e7-c654d16be3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:09.334048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb34bb-762f-4166-b3e7-c654d16be3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:09.358148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:09.430727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2beb34bb-762f-4166-b3e7-c654d16be3c5 None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:09.432066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2beb34bb-762f-4166-b3e7-c654d16be3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:09.558914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 671/1359] 10.222.0.78 () {58 vars in 1059 bytes} [Tue Aug 30 00:54:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:09.567330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef085b9d-c7ff-4d07-a448-6f48d80f45ef None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:09.569015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ef085b9d-c7ff-4d07-a448-6f48d80f45ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:09.731224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef085b9d-c7ff-4d07-a448-6f48d80f45ef None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:09.811362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:09.851747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc864d-4179-4dfb-9402-3969b20e30c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:09.852347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc864d-4179-4dfb-9402-3969b20e30c7 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:09.852620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc864d-4179-4dfb-9402-3969b20e30c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:09.852889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc864d-4179-4dfb-9402-3969b20e30c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:09.854493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-72fc864d-4179-4dfb-9402-3969b20e30c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:09.952196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc864d-4179-4dfb-9402-3969b20e30c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:09.954252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc864d-4179-4dfb-9402-3969b20e30c7 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:09.974920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 689/1360] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:12.141586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ed1f8-e979-4541-8bac-81a97cb2f677 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:12.142143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ed1f8-e979-4541-8bac-81a97cb2f677 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:12.142143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ed1f8-e979-4541-8bac-81a97cb2f677 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:12.143646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 672/1361] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:12.148694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd4ca7-5da8-4f56-9828-d32f1dc4ab29 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:12.149117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd4ca7-5da8-4f56-9828-d32f1dc4ab29 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:12.149274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd4ca7-5da8-4f56-9828-d32f1dc4ab29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:12.220032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5acd4ca7-5da8-4f56-9828-d32f1dc4ab29 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:12.221270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5acd4ca7-5da8-4f56-9828-d32f1dc4ab29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:12.271294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:12.351238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 690/1362] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:12.647701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ed1f8-e979-4541-8bac-81a97cb2f677 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:12.649258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cb2ed1f8-e979-4541-8bac-81a97cb2f677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:12.720541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:12.814139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ed1f8-e979-4541-8bac-81a97cb2f677 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:12.944565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31181431-64d9-4e71-aa38-bd9857ad10e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:12.945205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31181431-64d9-4e71-aa38-bd9857ad10e0 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:12.945490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31181431-64d9-4e71-aa38-bd9857ad10e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:12.945762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31181431-64d9-4e71-aa38-bd9857ad10e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:12.948630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-31181431-64d9-4e71-aa38-bd9857ad10e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:12.975361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:13.058683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31181431-64d9-4e71-aa38-bd9857ad10e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:13.060577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31181431-64d9-4e71-aa38-bd9857ad10e0 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:13.081928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 673/1363] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:15.350792 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181298b-767a-4460-8297-bc02dce46088 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:15.351339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181298b-767a-4460-8297-bc02dce46088 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:15.351339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181298b-767a-4460-8297-bc02dce46088 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:15.352678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 691/1364] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:54:15.358688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6612b4-c4d8-4a1d-b22e-6dba8e34f609 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:15.359126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6612b4-c4d8-4a1d-b22e-6dba8e34f609 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:15.359305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6612b4-c4d8-4a1d-b22e-6dba8e34f609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:15.428012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9d6612b4-c4d8-4a1d-b22e-6dba8e34f609 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:15.429246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9d6612b4-c4d8-4a1d-b22e-6dba8e34f609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:15.530336 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:15.541398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 674/1365] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:15.831047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01203f22-f751-4c86-b061-4362d3f0adcb None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:15.831451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01203f22-f751-4c86-b061-4362d3f0adcb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:15.832076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01203f22-f751-4c86-b061-4362d3f0adcb None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:15.833627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 692/1366] 10.222.0.78 () {54 vars in 962 bytes} [Tue Aug 30 00:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:15.838938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014abc0-364b-41cb-98e4-45b066613584 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:15.839222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014abc0-364b-41cb-98e4-45b066613584 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:15.839495 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014abc0-364b-41cb-98e4-45b066613584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:15.920712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4014abc0-364b-41cb-98e4-45b066613584 None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:15.921870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4014abc0-364b-41cb-98e4-45b066613584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:16.047173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 675/1367] 10.222.0.78 () {58 vars in 1059 bytes} [Tue Aug 30 00:54:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:16.055493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01203f22-f751-4c86-b061-4362d3f0adcb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:16.057455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-01203f22-f751-4c86-b061-4362d3f0adcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:16.216805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01203f22-f751-4c86-b061-4362d3f0adcb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:16.243051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:16.344768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365adf0-42fa-4be3-af4b-092a15a0f12a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:16.345469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365adf0-42fa-4be3-af4b-092a15a0f12a service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:16.345760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365adf0-42fa-4be3-af4b-092a15a0f12a service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:16.346019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365adf0-42fa-4be3-af4b-092a15a0f12a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:16.347616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f365adf0-42fa-4be3-af4b-092a15a0f12a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:16.445242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365adf0-42fa-4be3-af4b-092a15a0f12a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:16.447001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365adf0-42fa-4be3-af4b-092a15a0f12a service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:16.466629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 693/1368] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:17.016820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961851e-7b09-4ded-be73-704aa5928106 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:17.017423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961851e-7b09-4ded-be73-704aa5928106 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:17.017527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961851e-7b09-4ded-be73-704aa5928106 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:17.019062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 676/1369] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:54:17.024545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17238eb-6143-44de-a6ad-4c0241df4692 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:17.024837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17238eb-6143-44de-a6ad-4c0241df4692 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:17.025141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17238eb-6143-44de-a6ad-4c0241df4692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:17.058393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:17.096333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e17238eb-6143-44de-a6ad-4c0241df4692 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:17.097583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e17238eb-6143-44de-a6ad-4c0241df4692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:17.200502 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 694/1370] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:17.808855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf1416-e2a7-4d32-9811-c0f38db30a12 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:17.809284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf1416-e2a7-4d32-9811-c0f38db30a12 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:17.809665 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf1416-e2a7-4d32-9811-c0f38db30a12 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:17.812014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 677/1371] 10.222.0.78 () {54 vars in 964 bytes} [Tue Aug 30 00:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:17.816819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85904a-0495-48ee-a519-c824ee5f8fdb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:17.817310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85904a-0495-48ee-a519-c824ee5f8fdb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:17.817748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85904a-0495-48ee-a519-c824ee5f8fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:17.896831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2b85904a-0495-48ee-a519-c824ee5f8fdb None None] MFA Rules not processed for user `97e64ee3c8ee48c2ad8f351fb2a78bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:17.898231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2b85904a-0495-48ee-a519-c824ee5f8fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:17.948542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:17.982809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 695/1372] 10.222.0.78 () {58 vars in 1061 bytes} [Tue Aug 30 00:54:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:17.990241 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf1416-e2a7-4d32-9811-c0f38db30a12 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:17.991761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-35bf1416-e2a7-4d32-9811-c0f38db30a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:18.044599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:18.102453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf1416-e2a7-4d32-9811-c0f38db30a12 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:18.184985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c9ae1-36a7-479d-ac6d-8420b197424d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:18.186346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c9ae1-36a7-479d-ac6d-8420b197424d service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:18.186693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c9ae1-36a7-479d-ac6d-8420b197424d service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:18.187087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c9ae1-36a7-479d-ac6d-8420b197424d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:18.188894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-904c9ae1-36a7-479d-ac6d-8420b197424d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:18.310252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:18.325668 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c9ae1-36a7-479d-ac6d-8420b197424d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:18.327767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c9ae1-36a7-479d-ac6d-8420b197424d service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:18.348735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 678/1373] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:18.458573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cea78-98df-41f9-91d6-a6b10d359fd1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:18.458907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cea78-98df-41f9-91d6-a6b10d359fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:18.459179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cea78-98df-41f9-91d6-a6b10d359fd1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:18.460544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 696/1374] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:18.465754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac29277-a3ea-487f-a5eb-ce6e23c79f14 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:18.466129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac29277-a3ea-487f-a5eb-ce6e23c79f14 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:18.466409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac29277-a3ea-487f-a5eb-ce6e23c79f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:18.538218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7ac29277-a3ea-487f-a5eb-ce6e23c79f14 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:18.539445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7ac29277-a3ea-487f-a5eb-ce6e23c79f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:18.640368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 679/1375] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:18.937975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cea78-98df-41f9-91d6-a6b10d359fd1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:18.939645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ba1cea78-98df-41f9-91d6-a6b10d359fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:19.093510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cea78-98df-41f9-91d6-a6b10d359fd1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:19.146204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:19.209421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90ee66-08be-431a-a69f-58cb28f37c5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:19.209952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90ee66-08be-431a-a69f-58cb28f37c5f service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:19.210235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90ee66-08be-431a-a69f-58cb28f37c5f service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:19.210516 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90ee66-08be-431a-a69f-58cb28f37c5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:19.212111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9a90ee66-08be-431a-a69f-58cb28f37c5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:19.305307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90ee66-08be-431a-a69f-58cb28f37c5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:19.307277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90ee66-08be-431a-a69f-58cb28f37c5f service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:19.328308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 697/1376] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:21.282447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452097a-e43b-4150-8bc7-e614e5c72d0e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:21.282950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452097a-e43b-4150-8bc7-e614e5c72d0e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:21.283047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452097a-e43b-4150-8bc7-e614e5c72d0e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:21.284406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 680/1377] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:21.289540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc84498-8776-4af4-87d5-95a87d7db616 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:21.289938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc84498-8776-4af4-87d5-95a87d7db616 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:21.290077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc84498-8776-4af4-87d5-95a87d7db616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:21.365425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8cc84498-8776-4af4-87d5-95a87d7db616 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:21.366643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8cc84498-8776-4af4-87d5-95a87d7db616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:21.383994 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:21.492982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 698/1378] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:21.789928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452097a-e43b-4150-8bc7-e614e5c72d0e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:21.791643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7452097a-e43b-4150-8bc7-e614e5c72d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:21.874768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:21.990675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452097a-e43b-4150-8bc7-e614e5c72d0e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:22.116791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492250c9-6693-49c8-9fce-238b53907e58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:22.117487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492250c9-6693-49c8-9fce-238b53907e58 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:22.117879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492250c9-6693-49c8-9fce-238b53907e58 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:22.118229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492250c9-6693-49c8-9fce-238b53907e58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:22.119910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-492250c9-6693-49c8-9fce-238b53907e58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:22.140756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:22.214134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492250c9-6693-49c8-9fce-238b53907e58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:22.215900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492250c9-6693-49c8-9fce-238b53907e58 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:22.234675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 681/1379] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:24.227183 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622af68b-5635-4160-b000-a7d16f5fb90f None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:24.227702 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622af68b-5635-4160-b000-a7d16f5fb90f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:24.227958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622af68b-5635-4160-b000-a7d16f5fb90f None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:24.229782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 699/1380] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:24.235548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19c3a1-f475-4fca-8de7-cc460a7560db None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:24.235977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19c3a1-f475-4fca-8de7-cc460a7560db None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:24.236164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19c3a1-f475-4fca-8de7-cc460a7560db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:24.317224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6d19c3a1-f475-4fca-8de7-cc460a7560db None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:24.318435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6d19c3a1-f475-4fca-8de7-cc460a7560db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:24.415888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:24.424276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 682/1381] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:24.729752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622af68b-5635-4160-b000-a7d16f5fb90f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:24.731514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-622af68b-5635-4160-b000-a7d16f5fb90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:24.895809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:24.914290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622af68b-5635-4160-b000-a7d16f5fb90f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:25.047688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3823d6b-d711-49c4-8bcf-15e14a71a3c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:25.048367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3823d6b-d711-49c4-8bcf-15e14a71a3c7 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:25.048747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3823d6b-d711-49c4-8bcf-15e14a71a3c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:25.049213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3823d6b-d711-49c4-8bcf-15e14a71a3c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:25.050952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b3823d6b-d711-49c4-8bcf-15e14a71a3c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:25.146155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3823d6b-d711-49c4-8bcf-15e14a71a3c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:25.147903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3823d6b-d711-49c4-8bcf-15e14a71a3c7 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:25.159000 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:25.168286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 700/1382] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:54:27.690250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77023373-5552-4a39-9fa8-4d3dd7412007 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:27.690844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77023373-5552-4a39-9fa8-4d3dd7412007 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:27.690844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77023373-5552-4a39-9fa8-4d3dd7412007 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:27.692771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 683/1383] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:27.699998 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff022e-2ceb-4874-81cd-7cfb3afe395e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:27.700497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff022e-2ceb-4874-81cd-7cfb3afe395e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:27.701466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff022e-2ceb-4874-81cd-7cfb3afe395e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:27.780214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7bff022e-2ceb-4874-81cd-7cfb3afe395e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:27.781869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7bff022e-2ceb-4874-81cd-7cfb3afe395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:27.897871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 701/1384] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:28.190630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77023373-5552-4a39-9fa8-4d3dd7412007 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:28.192970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-77023373-5552-4a39-9fa8-4d3dd7412007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:28.352833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77023373-5552-4a39-9fa8-4d3dd7412007 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:28.443566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:28.504004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb16b7-f958-47fd-889d-eaa3d3f8e72e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:28.505009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb16b7-f958-47fd-889d-eaa3d3f8e72e service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:28.505379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb16b7-f958-47fd-889d-eaa3d3f8e72e service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:28.505724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb16b7-f958-47fd-889d-eaa3d3f8e72e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:28.507973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-08fb16b7-f958-47fd-889d-eaa3d3f8e72e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:28.611554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb16b7-f958-47fd-889d-eaa3d3f8e72e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:28.613208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb16b7-f958-47fd-889d-eaa3d3f8e72e service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:28.632460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 684/1385] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:30.830480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59505340-53a7-4b74-bccb-2550e5fcbcf1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:30.831016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59505340-53a7-4b74-bccb-2550e5fcbcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:30.831152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59505340-53a7-4b74-bccb-2550e5fcbcf1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:30.832667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 702/1386] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:30.838537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a00b8-9e23-4bae-97a1-59dd69dfdbfa None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:30.838949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a00b8-9e23-4bae-97a1-59dd69dfdbfa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:30.839120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a00b8-9e23-4bae-97a1-59dd69dfdbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:30.913983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f01a00b8-9e23-4bae-97a1-59dd69dfdbfa None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:30.915366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f01a00b8-9e23-4bae-97a1-59dd69dfdbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:30.954181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:31.029635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 685/1387] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:31.328800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59505340-53a7-4b74-bccb-2550e5fcbcf1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:31.330365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-59505340-53a7-4b74-bccb-2550e5fcbcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:31.411352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:31.493409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59505340-53a7-4b74-bccb-2550e5fcbcf1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:31.626222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370301d-4dca-434e-9a13-ed7f697b089c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:31.626835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370301d-4dca-434e-9a13-ed7f697b089c service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:31.627112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370301d-4dca-434e-9a13-ed7f697b089c service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:31.627379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370301d-4dca-434e-9a13-ed7f697b089c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:31.628980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1370301d-4dca-434e-9a13-ed7f697b089c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:31.674293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:31.738244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370301d-4dca-434e-9a13-ed7f697b089c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:31.740186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370301d-4dca-434e-9a13-ed7f697b089c service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:31.763589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 703/1388] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:33.669309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0043b-ecf9-44fa-8413-d91a1f77bba7 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:33.669896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0043b-ecf9-44fa-8413-d91a1f77bba7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:33.669896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0043b-ecf9-44fa-8413-d91a1f77bba7 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:33.671207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 686/1389] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:33.676527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189badd-4397-4a74-aa55-6bc79e652a05 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:33.676957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189badd-4397-4a74-aa55-6bc79e652a05 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:33.677095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189badd-4397-4a74-aa55-6bc79e652a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:33.739713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f0aa8-0c4e-4840-afca-e2f81a3a8f4d None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:33.739990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f0aa8-0c4e-4840-afca-e2f81a3a8f4d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:33.740256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f0aa8-0c4e-4840-afca-e2f81a3a8f4d None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:33.741625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 687/1390] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:54:33.747544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faece8-5139-4776-bb40-60ebefc74d83 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:33.747942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faece8-5139-4776-bb40-60ebefc74d83 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:33.748218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faece8-5139-4776-bb40-60ebefc74d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:33.748474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e189badd-4397-4a74-aa55-6bc79e652a05 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:33.749879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e189badd-4397-4a74-aa55-6bc79e652a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:33.815520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d8faece8-5139-4776-bb40-60ebefc74d83 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:33.816728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d8faece8-5139-4776-bb40-60ebefc74d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:33.848179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 704/1391] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:33.916024 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 688/1392] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:54:34.150650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988af1-e31a-4ae4-87b8-017538551caf None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:34.150956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988af1-e31a-4ae4-87b8-017538551caf None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:34.151193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988af1-e31a-4ae4-87b8-017538551caf None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:34.152703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 705/1393] 10.222.0.78 () {54 vars in 967 bytes} [Tue Aug 30 00:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:34.158419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e5e45-d81e-4b6d-a8be-c819c0b9df3d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:34.158694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e5e45-d81e-4b6d-a8be-c819c0b9df3d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:34.158965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e5e45-d81e-4b6d-a8be-c819c0b9df3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:34.176144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:34.215194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0988af1-e31a-4ae4-87b8-017538551caf None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:34.217421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e0988af1-e31a-4ae4-87b8-017538551caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:34.235750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:34.245809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ff1e5e45-d81e-4b6d-a8be-c819c0b9df3d None None] MFA Rules not processed for user `77a47e77cc1942618a3584862042c5f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:34.247605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ff1e5e45-d81e-4b6d-a8be-c819c0b9df3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:34.345748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 689/1394] 10.222.0.78 () {58 vars in 1064 bytes} [Tue Aug 30 00:54:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:34.353713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e5e45-d81e-4b6d-a8be-c819c0b9df3d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:34.362093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e5e45-d81e-4b6d-a8be-c819c0b9df3d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:34.391532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0988af1-e31a-4ae4-87b8-017538551caf None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:34.438357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339ec43-ef5c-4b21-a091-2d39231de715 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a47e77cc1942618a3584862042c5f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:34.438876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339ec43-ef5c-4b21-a091-2d39231de715 service glance] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:34.439153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339ec43-ef5c-4b21-a091-2d39231de715 service glance] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:34.439422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339ec43-ef5c-4b21-a091-2d39231de715 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:34.441066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5339ec43-ef5c-4b21-a091-2d39231de715 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:34.471610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:34.484319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:34.511969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f6565-2365-44ef-a30c-690f2616ab20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:34.512841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f6565-2365-44ef-a30c-690f2616ab20 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:34.513143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f6565-2365-44ef-a30c-690f2616ab20 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:34.513444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f6565-2365-44ef-a30c-690f2616ab20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:34.515043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-349f6565-2365-44ef-a30c-690f2616ab20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:34.552840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339ec43-ef5c-4b21-a091-2d39231de715 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:34.554330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339ec43-ef5c-4b21-a091-2d39231de715 service glance] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:34.573342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 690/1395] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:54:34.607634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f6565-2365-44ef-a30c-690f2616ab20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:34.609274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f6565-2365-44ef-a30c-690f2616ab20 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:34.627501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 706/1396] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:54:35.337179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b723e7-3b7a-4caa-9c2b-4b65e2a4d6a7 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:35.337696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b723e7-3b7a-4caa-9c2b-4b65e2a4d6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:35.337791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b723e7-3b7a-4caa-9c2b-4b65e2a4d6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:35.406462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c6b723e7-3b7a-4caa-9c2b-4b65e2a4d6a7 None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:35.407672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c6b723e7-3b7a-4caa-9c2b-4b65e2a4d6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:35.492907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:35.499462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 691/1397] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 00:54:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:35.541939 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537d094-bc3e-4d6f-a09c-7f6514b205c9 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:35.542426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537d094-bc3e-4d6f-a09c-7f6514b205c9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:35.542534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537d094-bc3e-4d6f-a09c-7f6514b205c9 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:35.544468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 707/1398] 10.222.0.78 () {54 vars in 943 bytes} [Tue Aug 30 00:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:35.551085 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b858ac-2d0e-4d3a-a57f-df19e785cf00 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:35.551497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b858ac-2d0e-4d3a-a57f-df19e785cf00 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:35.551809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b858ac-2d0e-4d3a-a57f-df19e785cf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:35.630860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a7b858ac-2d0e-4d3a-a57f-df19e785cf00 None None] MFA Rules not processed for user `c53b9bcd675940c19febbc8fefb1f28b`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:35.632068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a7b858ac-2d0e-4d3a-a57f-df19e785cf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:35.744138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 692/1399] 10.222.0.78 () {58 vars in 1040 bytes} [Tue Aug 30 00:54:35 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:35.754465 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537d094-bc3e-4d6f-a09c-7f6514b205c9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:35.756017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8537d094-bc3e-4d6f-a09c-7f6514b205c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:35.860021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:35.903009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537d094-bc3e-4d6f-a09c-7f6514b205c9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:36.007208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad57fc2-3bb2-483c-a221-b480392195b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:36.007809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad57fc2-3bb2-483c-a221-b480392195b1 service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:36.008085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad57fc2-3bb2-483c-a221-b480392195b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:36.008347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad57fc2-3bb2-483c-a221-b480392195b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:36.010004 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-aad57fc2-3bb2-483c-a221-b480392195b1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:36.093562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad57fc2-3bb2-483c-a221-b480392195b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:36.095170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad57fc2-3bb2-483c-a221-b480392195b1 service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:36.098661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:36.104005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 708/1400] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 00:54:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:54:36.611042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3336c8-c6cf-47d6-9886-35efd8e20784 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:36.611602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3336c8-c6cf-47d6-9886-35efd8e20784 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:36.611602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3336c8-c6cf-47d6-9886-35efd8e20784 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:36.612928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 693/1401] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:36.618145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e256f-3866-4983-84bc-bd3a610712ed None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:36.618575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e256f-3866-4983-84bc-bd3a610712ed None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:36.618897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e256f-3866-4983-84bc-bd3a610712ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:36.690316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-644e256f-3866-4983-84bc-bd3a610712ed None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:36.691820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-644e256f-3866-4983-84bc-bd3a610712ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:36.799861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 709/1402] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:37.092349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3336c8-c6cf-47d6-9886-35efd8e20784 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:37.094624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ec3336c8-c6cf-47d6-9886-35efd8e20784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:37.162255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:37.266237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3336c8-c6cf-47d6-9886-35efd8e20784 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:37.400243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb302085-c2a2-4dda-b34c-6dd052d5ac11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:37.400849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb302085-c2a2-4dda-b34c-6dd052d5ac11 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:37.401160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb302085-c2a2-4dda-b34c-6dd052d5ac11 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:37.401437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb302085-c2a2-4dda-b34c-6dd052d5ac11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:37.403615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fb302085-c2a2-4dda-b34c-6dd052d5ac11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:37.422786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:37.507252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb302085-c2a2-4dda-b34c-6dd052d5ac11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:37.509287 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb302085-c2a2-4dda-b34c-6dd052d5ac11 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:37.529801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 694/1403] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:39.539151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6a7ba-447a-43fa-ac02-ef84d92e207e None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:39.540004 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6a7ba-447a-43fa-ac02-ef84d92e207e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:39.540282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6a7ba-447a-43fa-ac02-ef84d92e207e None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:39.541945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 710/1404] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:39.547727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f128be-2f0b-4e17-bad5-ad0f1ddb0cb3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:39.548177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f128be-2f0b-4e17-bad5-ad0f1ddb0cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:39.548407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f128be-2f0b-4e17-bad5-ad0f1ddb0cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:39.625507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-59f128be-2f0b-4e17-bad5-ad0f1ddb0cb3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:39.626715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-59f128be-2f0b-4e17-bad5-ad0f1ddb0cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:39.765983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:39.774117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 695/1405] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:40.075964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6a7ba-447a-43fa-ac02-ef84d92e207e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:40.077688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7cf6a7ba-447a-43fa-ac02-ef84d92e207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:40.167403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:40.249789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6a7ba-447a-43fa-ac02-ef84d92e207e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:40.390356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660b930-34b1-4f22-8e0e-e43703cc9462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:40.390356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660b930-34b1-4f22-8e0e-e43703cc9462 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:40.390356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660b930-34b1-4f22-8e0e-e43703cc9462 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:40.390356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660b930-34b1-4f22-8e0e-e43703cc9462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:40.390829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a660b930-34b1-4f22-8e0e-e43703cc9462 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:40.432647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:40.489212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660b930-34b1-4f22-8e0e-e43703cc9462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:40.490815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660b930-34b1-4f22-8e0e-e43703cc9462 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:40.509786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 711/1406] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:42.498549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185345d-8db2-4873-a776-c270d0d7e3ba None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:42.499103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185345d-8db2-4873-a776-c270d0d7e3ba None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:42.499103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185345d-8db2-4873-a776-c270d0d7e3ba None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:42.504757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 696/1407] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:42.506784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858697a2-e0a4-4c05-b86f-4a96969add97 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:42.506784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858697a2-e0a4-4c05-b86f-4a96969add97 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:42.507337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858697a2-e0a4-4c05-b86f-4a96969add97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:42.583092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-858697a2-e0a4-4c05-b86f-4a96969add97 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:42.584217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-858697a2-e0a4-4c05-b86f-4a96969add97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:42.697116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 712/1408] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:42.985925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185345d-8db2-4873-a776-c270d0d7e3ba None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:42.990067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e185345d-8db2-4873-a776-c270d0d7e3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:43.148544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185345d-8db2-4873-a776-c270d0d7e3ba None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:43.224260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:43.291515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e4b46-676e-4d3b-8237-be9e55bc6fb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:43.292128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e4b46-676e-4d3b-8237-be9e55bc6fb5 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:43.292409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e4b46-676e-4d3b-8237-be9e55bc6fb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:43.292680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e4b46-676e-4d3b-8237-be9e55bc6fb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:43.295089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-566e4b46-676e-4d3b-8237-be9e55bc6fb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:43.405995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e4b46-676e-4d3b-8237-be9e55bc6fb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:43.407892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e4b46-676e-4d3b-8237-be9e55bc6fb5 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:43.427586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 697/1409] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:52.692805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=93 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:53.425315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=65 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:59.016040 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e828fb-7279-461d-aa24-6009e9ecb4b1 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:59.016663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e828fb-7279-461d-aa24-6009e9ecb4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:59.016663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e828fb-7279-461d-aa24-6009e9ecb4b1 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:59.017954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 713/1410] 10.222.0.78 () {54 vars in 918 bytes} [Tue Aug 30 00:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:59.023035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c0f56-eeb3-4972-a60a-87dff27c7793 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:59.023457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c0f56-eeb3-4972-a60a-87dff27c7793 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:59.023559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c0f56-eeb3-4972-a60a-87dff27c7793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:59.101759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-242c0f56-eeb3-4972-a60a-87dff27c7793 None None] MFA Rules not processed for user `8f860912e1f0406cadd70da440b2ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:59.102966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-242c0f56-eeb3-4972-a60a-87dff27c7793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:59.208114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 698/1411] 10.222.0.78 () {58 vars in 1015 bytes} [Tue Aug 30 00:54:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:59.222859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8f514-e563-475e-82c1-95c53199df27 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:59.223175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8f514-e563-475e-82c1-95c53199df27 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:59.223450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8f514-e563-475e-82c1-95c53199df27 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:59.224824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 714/1412] 10.222.0.78 () {54 vars in 918 bytes} [Tue Aug 30 00:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:59.229273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4cfa2-8c85-41d5-af53-19c0b4cb7c34 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:59.229563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4cfa2-8c85-41d5-af53-19c0b4cb7c34 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:59.230439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4cfa2-8c85-41d5-af53-19c0b4cb7c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:59.304389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:59.307774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-00b4cfa2-8c85-41d5-af53-19c0b4cb7c34 None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:59.309568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-00b4cfa2-8c85-41d5-af53-19c0b4cb7c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:59.433448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 699/1413] 10.222.0.78 () {58 vars in 1015 bytes} [Tue Aug 30 00:54:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:01.851164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e949c16-42a6-4675-affe-2e41cb9194d9 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:01.851709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e949c16-42a6-4675-affe-2e41cb9194d9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:01.851709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e949c16-42a6-4675-affe-2e41cb9194d9 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:01.853026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:01.859019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f759e-9cd1-418a-828f-8939e65d5122 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:01.859424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f759e-9cd1-418a-828f-8939e65d5122 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:01.859574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f759e-9cd1-418a-828f-8939e65d5122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:01.926000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c12f759e-9cd1-418a-828f-8939e65d5122 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:01.927206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c12f759e-9cd1-418a-828f-8939e65d5122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:01.982334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:02.024869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:02.320974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e949c16-42a6-4675-affe-2e41cb9194d9 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:02.322534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8e949c16-42a6-4675-affe-2e41cb9194d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:02.469961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e949c16-42a6-4675-affe-2e41cb9194d9 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:02.541558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:02.584271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6fd1c-3f84-47cf-a4c9-5e0d489d506d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:02.584875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6fd1c-3f84-47cf-a4c9-5e0d489d506d service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:02.585251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6fd1c-3f84-47cf-a4c9-5e0d489d506d service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:02.585519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6fd1c-3f84-47cf-a4c9-5e0d489d506d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:02.587104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-26f6fd1c-3f84-47cf-a4c9-5e0d489d506d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:02.674881 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6fd1c-3f84-47cf-a4c9-5e0d489d506d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:02.676459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6fd1c-3f84-47cf-a4c9-5e0d489d506d service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:02.694156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 716/1416] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:03.591678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8775155-b25c-4a02-81e3-fa804af03418 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:03.592216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8775155-b25c-4a02-81e3-fa804af03418 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:03.592216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8775155-b25c-4a02-81e3-fa804af03418 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:03.593670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:03.598600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17723a-b384-488c-954d-a03f0d066ffa None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:03.599031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17723a-b384-488c-954d-a03f0d066ffa None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:03.599138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17723a-b384-488c-954d-a03f0d066ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:03.673717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5a17723a-b384-488c-954d-a03f0d066ffa None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:03.674922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5a17723a-b384-488c-954d-a03f0d066ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:03.703033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:03.772226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:03.923377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8775155-b25c-4a02-81e3-fa804af03418 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:03.924937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f8775155-b25c-4a02-81e3-fa804af03418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:04.069189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8775155-b25c-4a02-81e3-fa804af03418 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:04.109559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:04.182332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcdc63-b38b-484e-b1bf-63b959172b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:04.183462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcdc63-b38b-484e-b1bf-63b959172b16 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:04.183735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcdc63-b38b-484e-b1bf-63b959172b16 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:04.183998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcdc63-b38b-484e-b1bf-63b959172b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:04.185666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-dabcdc63-b38b-484e-b1bf-63b959172b16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:04.273357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcdc63-b38b-484e-b1bf-63b959172b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:04.274936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcdc63-b38b-484e-b1bf-63b959172b16 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:04.295388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 702/1419] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:05.194954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b23f02-c0b4-4b8a-b7b7-91d4a3d3376d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:05.195531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b23f02-c0b4-4b8a-b7b7-91d4a3d3376d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:05.195531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b23f02-c0b4-4b8a-b7b7-91d4a3d3376d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:05.196848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:05.201917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ac9a2-94c2-4ece-98a4-02c578fea01e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:05.202314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ac9a2-94c2-4ece-98a4-02c578fea01e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:05.202468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ac9a2-94c2-4ece-98a4-02c578fea01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:05.258435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:05.282302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9e7ac9a2-94c2-4ece-98a4-02c578fea01e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:05.283553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9e7ac9a2-94c2-4ece-98a4-02c578fea01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:05.381911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:05.534725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b23f02-c0b4-4b8a-b7b7-91d4a3d3376d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:05.536186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-23b23f02-c0b4-4b8a-b7b7-91d4a3d3376d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:05.682022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b23f02-c0b4-4b8a-b7b7-91d4a3d3376d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:05.696695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:05.799978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfdd42-6bdb-4498-b522-25b88d220147 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:05.800535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfdd42-6bdb-4498-b522-25b88d220147 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:05.800812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfdd42-6bdb-4498-b522-25b88d220147 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:05.801093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfdd42-6bdb-4498-b522-25b88d220147 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:05.802702 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-02bfdd42-6bdb-4498-b522-25b88d220147 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:05.890647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfdd42-6bdb-4498-b522-25b88d220147 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:05.892276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfdd42-6bdb-4498-b522-25b88d220147 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:05.910309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 719/1422] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:06.771946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9072dc1-89e6-40fe-b59e-3f76b3c8f409 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:06.772490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9072dc1-89e6-40fe-b59e-3f76b3c8f409 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:06.772490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9072dc1-89e6-40fe-b59e-3f76b3c8f409 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:06.773884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:06.779392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9de2-8cab-4e3e-946c-658632600d87 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:06.779793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9de2-8cab-4e3e-946c-658632600d87 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:06.779929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9de2-8cab-4e3e-946c-658632600d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:06.799681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:06.855759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0a8b9de2-8cab-4e3e-946c-658632600d87 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:06.856941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0a8b9de2-8cab-4e3e-946c-658632600d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:06.951060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:07.103203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9072dc1-89e6-40fe-b59e-3f76b3c8f409 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:07.104866 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f9072dc1-89e6-40fe-b59e-3f76b3c8f409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:07.211530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:07.247130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9072dc1-89e6-40fe-b59e-3f76b3c8f409 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:07.358942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f5f81-c2f2-4434-bc34-e5034bfbb659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:07.359457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f5f81-c2f2-4434-bc34-e5034bfbb659 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:07.359731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f5f81-c2f2-4434-bc34-e5034bfbb659 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:07.359992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f5f81-c2f2-4434-bc34-e5034bfbb659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:07.361606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-037f5f81-c2f2-4434-bc34-e5034bfbb659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:07.439470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:07.454532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f5f81-c2f2-4434-bc34-e5034bfbb659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:07.456120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f5f81-c2f2-4434-bc34-e5034bfbb659 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:07.474129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 705/1425] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:08.351616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ebc3e-4374-430b-b0b5-d8e3cacddaec None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:08.352171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ebc3e-4374-430b-b0b5-d8e3cacddaec None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:08.352171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ebc3e-4374-430b-b0b5-d8e3cacddaec None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:08.353575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:08.359046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e10f8-a6d4-4f1f-b734-d8d3d52f8b36 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:08.359589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e10f8-a6d4-4f1f-b734-d8d3d52f8b36 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:08.359589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e10f8-a6d4-4f1f-b734-d8d3d52f8b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:08.436913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7a8e10f8-a6d4-4f1f-b734-d8d3d52f8b36 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:08.438158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7a8e10f8-a6d4-4f1f-b734-d8d3d52f8b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:08.538390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:08.690567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ebc3e-4374-430b-b0b5-d8e3cacddaec None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:08.692075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7e8ebc3e-4374-430b-b0b5-d8e3cacddaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:08.774586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:08.836589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ebc3e-4374-430b-b0b5-d8e3cacddaec None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:08.951521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b7dc7-a4ea-4ff1-9477-bd931c79f7c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:08.952118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7dc7-a4ea-4ff1-9477-bd931c79f7c3 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:08.952391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7dc7-a4ea-4ff1-9477-bd931c79f7c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:08.952659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7dc7-a4ea-4ff1-9477-bd931c79f7c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:08.954273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fa5b7dc7-a4ea-4ff1-9477-bd931c79f7c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:09.003158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:09.044880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b7dc7-a4ea-4ff1-9477-bd931c79f7c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:09.046517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b7dc7-a4ea-4ff1-9477-bd931c79f7c3 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:09.065346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 722/1428] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:09.875381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c7010-43c7-47a8-91a6-152ed2edcf97 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:09.875959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c7010-43c7-47a8-91a6-152ed2edcf97 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:09.875959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c7010-43c7-47a8-91a6-152ed2edcf97 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:09.877366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:09.882470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6d61f-07a4-42c4-bafc-16b59e19f9c4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:09.882883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6d61f-07a4-42c4-bafc-16b59e19f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:09.883048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6d61f-07a4-42c4-bafc-16b59e19f9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:09.958650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e0b6d61f-07a4-42c4-bafc-16b59e19f9c4 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:09.959872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e0b6d61f-07a4-42c4-bafc-16b59e19f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:10.055603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:10.207029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c7010-43c7-47a8-91a6-152ed2edcf97 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:10.208468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-281c7010-43c7-47a8-91a6-152ed2edcf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:10.256289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:10.355278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c7010-43c7-47a8-91a6-152ed2edcf97 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:10.468590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698c2d4-0461-48f5-9cf3-662354450c6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:10.469138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c2d4-0461-48f5-9cf3-662354450c6e service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:10.469423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c2d4-0461-48f5-9cf3-662354450c6e service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:10.469692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c2d4-0461-48f5-9cf3-662354450c6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:10.471303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1698c2d4-0461-48f5-9cf3-662354450c6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:10.478258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:10.567466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698c2d4-0461-48f5-9cf3-662354450c6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:10.569125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698c2d4-0461-48f5-9cf3-662354450c6e service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:10.588096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 708/1431] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:11.449900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25f1a9-d971-4e85-a042-3af67095fa19 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:11.450483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25f1a9-d971-4e85-a042-3af67095fa19 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:11.450483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25f1a9-d971-4e85-a042-3af67095fa19 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:11.451816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:11.456669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daf9e2-b321-44e3-b36f-da78932058d0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:11.457055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daf9e2-b321-44e3-b36f-da78932058d0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:11.457248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daf9e2-b321-44e3-b36f-da78932058d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:11.539066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b8daf9e2-b321-44e3-b36f-da78932058d0 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:11.540195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b8daf9e2-b321-44e3-b36f-da78932058d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:11.613698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:11.643329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:11.795671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25f1a9-d971-4e85-a042-3af67095fa19 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:11.797145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0d25f1a9-d971-4e85-a042-3af67095fa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:11.815254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:11.943374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25f1a9-d971-4e85-a042-3af67095fa19 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:12.031151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:12.056372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1afb7f-b18c-47a4-a737-88acd9faccc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:12.056906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1afb7f-b18c-47a4-a737-88acd9faccc3 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:12.057206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1afb7f-b18c-47a4-a737-88acd9faccc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:12.057472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1afb7f-b18c-47a4-a737-88acd9faccc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:12.059554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1c1afb7f-b18c-47a4-a737-88acd9faccc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:12.146382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1afb7f-b18c-47a4-a737-88acd9faccc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:12.147962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1afb7f-b18c-47a4-a737-88acd9faccc3 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:12.165512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 725/1434] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:12.970303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32109965-ee8c-45d3-b936-d0fcdf50007e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:12.970887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32109965-ee8c-45d3-b936-d0fcdf50007e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:12.970887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32109965-ee8c-45d3-b936-d0fcdf50007e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:12.972202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:12.976981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0de00e-643f-4b18-bad3-051d55e9f726 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:12.977435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0de00e-643f-4b18-bad3-051d55e9f726 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:12.977626 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0de00e-643f-4b18-bad3-051d55e9f726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:13.052164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ab0de00e-643f-4b18-bad3-051d55e9f726 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:13.053400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ab0de00e-643f-4b18-bad3-051d55e9f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:13.094349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:13.148208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:13.299965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32109965-ee8c-45d3-b936-d0fcdf50007e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:13.301528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-32109965-ee8c-45d3-b936-d0fcdf50007e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:13.447830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32109965-ee8c-45d3-b936-d0fcdf50007e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:13.501744 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:13.562323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096cb1-91ad-41cf-9687-6002dbb25767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:13.562831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096cb1-91ad-41cf-9687-6002dbb25767 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:13.563215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096cb1-91ad-41cf-9687-6002dbb25767 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:13.563486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096cb1-91ad-41cf-9687-6002dbb25767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:13.565266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f8096cb1-91ad-41cf-9687-6002dbb25767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:13.653947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096cb1-91ad-41cf-9687-6002dbb25767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:13.655870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096cb1-91ad-41cf-9687-6002dbb25767 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:13.673986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 711/1437] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:14.491573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca7e69-f514-449a-930a-b04dc0ae81ac None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:14.492116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca7e69-f514-449a-930a-b04dc0ae81ac None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:14.492116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca7e69-f514-449a-930a-b04dc0ae81ac None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:14.493417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:14.498781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2bf16-6df1-407b-92fc-10c0943e23fe None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:14.499182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2bf16-6df1-407b-92fc-10c0943e23fe None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:14.499325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2bf16-6df1-407b-92fc-10c0943e23fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:14.577752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-63c2bf16-6df1-407b-92fc-10c0943e23fe None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:14.578961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-63c2bf16-6df1-407b-92fc-10c0943e23fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:14.580948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:14.674944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:14.826583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca7e69-f514-449a-930a-b04dc0ae81ac None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:14.828015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-44ca7e69-f514-449a-930a-b04dc0ae81ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:14.974476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca7e69-f514-449a-930a-b04dc0ae81ac None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:15.001322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:15.087698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556405a2-1253-42c2-8f1f-ddd61ef2dbec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:15.088212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556405a2-1253-42c2-8f1f-ddd61ef2dbec service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:15.088484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556405a2-1253-42c2-8f1f-ddd61ef2dbec service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:15.088780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556405a2-1253-42c2-8f1f-ddd61ef2dbec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:15.090574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-556405a2-1253-42c2-8f1f-ddd61ef2dbec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:15.180406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556405a2-1253-42c2-8f1f-ddd61ef2dbec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:15.182021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556405a2-1253-42c2-8f1f-ddd61ef2dbec service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:15.200774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 728/1440] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:17.065881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5712e-7be9-4ea4-af83-bcb8bacf6110 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:17.066478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5712e-7be9-4ea4-af83-bcb8bacf6110 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:17.066478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5712e-7be9-4ea4-af83-bcb8bacf6110 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:17.067787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:17.073049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6200957-925f-4f12-a5dc-93c64a709375 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:17.074063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6200957-925f-4f12-a5dc-93c64a709375 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:17.074170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6200957-925f-4f12-a5dc-93c64a709375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:17.115132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:17.141478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d6200957-925f-4f12-a5dc-93c64a709375 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:17.142689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d6200957-925f-4f12-a5dc-93c64a709375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:17.240099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:17.533415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5712e-7be9-4ea4-af83-bcb8bacf6110 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:17.534965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-14a5712e-7be9-4ea4-af83-bcb8bacf6110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:17.643399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5712e-7be9-4ea4-af83-bcb8bacf6110 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:17.676169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:17.720366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67154693-afe3-4ea0-9fcc-7d09aeb82b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:17.720966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67154693-afe3-4ea0-9fcc-7d09aeb82b47 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:17.721282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67154693-afe3-4ea0-9fcc-7d09aeb82b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:17.721554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67154693-afe3-4ea0-9fcc-7d09aeb82b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:17.723143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-67154693-afe3-4ea0-9fcc-7d09aeb82b47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:17.831433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67154693-afe3-4ea0-9fcc-7d09aeb82b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:17.832777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67154693-afe3-4ea0-9fcc-7d09aeb82b47 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:17.852312 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 714/1443] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:19.922858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231aaef9-5b09-45af-a418-fa626c51d89a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:19.923437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231aaef9-5b09-45af-a418-fa626c51d89a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:19.923437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231aaef9-5b09-45af-a418-fa626c51d89a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:19.924698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:19.930049 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce50bfb-667c-4288-a695-66b0103011a0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:19.930753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce50bfb-667c-4288-a695-66b0103011a0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:19.930861 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce50bfb-667c-4288-a695-66b0103011a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:19.999661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6ce50bfb-667c-4288-a695-66b0103011a0 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:20.000874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6ce50bfb-667c-4288-a695-66b0103011a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:20.014373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:20.218485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:20.501519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231aaef9-5b09-45af-a418-fa626c51d89a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:20.503067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-231aaef9-5b09-45af-a418-fa626c51d89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:20.606126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231aaef9-5b09-45af-a418-fa626c51d89a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:20.618542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:20.679497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f29b00-7747-4ff9-9c72-637a26ebd429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:20.680338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f29b00-7747-4ff9-9c72-637a26ebd429 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:20.680621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f29b00-7747-4ff9-9c72-637a26ebd429 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:20.680884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f29b00-7747-4ff9-9c72-637a26ebd429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:20.682527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-68f29b00-7747-4ff9-9c72-637a26ebd429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:20.792643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f29b00-7747-4ff9-9c72-637a26ebd429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:20.794094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f29b00-7747-4ff9-9c72-637a26ebd429 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:20.929972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 731/1446] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:23.176100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404281d-8fbf-4a40-9d6f-ddca8d04c752 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:23.176648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404281d-8fbf-4a40-9d6f-ddca8d04c752 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:23.176648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404281d-8fbf-4a40-9d6f-ddca8d04c752 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:23.178038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:23.184036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd42d5-5f6d-44c5-a0bf-34cce6595fa9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:23.184445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd42d5-5f6d-44c5-a0bf-34cce6595fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:23.184576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd42d5-5f6d-44c5-a0bf-34cce6595fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:23.231752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:23.250882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3fbd42d5-5f6d-44c5-a0bf-34cce6595fa9 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:23.252080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3fbd42d5-5f6d-44c5-a0bf-34cce6595fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:23.348155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:23.632487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404281d-8fbf-4a40-9d6f-ddca8d04c752 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:23.634043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9404281d-8fbf-4a40-9d6f-ddca8d04c752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:23.745064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404281d-8fbf-4a40-9d6f-ddca8d04c752 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:23.785392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:23.819121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aaca0-769f-40bd-aa23-e2456c1e99c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:23.819709 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aaca0-769f-40bd-aa23-e2456c1e99c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:23.819998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aaca0-769f-40bd-aa23-e2456c1e99c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:23.820247 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aaca0-769f-40bd-aa23-e2456c1e99c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:23.821849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c95aaca0-769f-40bd-aa23-e2456c1e99c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:23.927146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95aaca0-769f-40bd-aa23-e2456c1e99c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:23.928479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95aaca0-769f-40bd-aa23-e2456c1e99c1 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:23.946598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 717/1449] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:25.974728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babbae6-4f8c-4dcc-9004-dd1a7170065d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:25.975211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babbae6-4f8c-4dcc-9004-dd1a7170065d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:25.975313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babbae6-4f8c-4dcc-9004-dd1a7170065d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:25.976694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:25.982405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac44c0-d830-4f18-8933-4cebbcfb1a4c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:25.982830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac44c0-d830-4f18-8933-4cebbcfb1a4c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:25.982957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac44c0-d830-4f18-8933-4cebbcfb1a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:26.048349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-56ac44c0-d830-4f18-8933-4cebbcfb1a4c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:26.049586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-56ac44c0-d830-4f18-8933-4cebbcfb1a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:26.070178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:26.147339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:26.439218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babbae6-4f8c-4dcc-9004-dd1a7170065d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:26.440811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7babbae6-4f8c-4dcc-9004-dd1a7170065d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:26.546146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babbae6-4f8c-4dcc-9004-dd1a7170065d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:26.567006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:26.619651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f0e47-e61e-495e-8781-79a090dd1d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:26.620997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f0e47-e61e-495e-8781-79a090dd1d7c service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:26.621298 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f0e47-e61e-495e-8781-79a090dd1d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:26.621562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f0e47-e61e-495e-8781-79a090dd1d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:26.623146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-878f0e47-e61e-495e-8781-79a090dd1d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:26.733376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f0e47-e61e-495e-8781-79a090dd1d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:26.734792 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f0e47-e61e-495e-8781-79a090dd1d7c service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:26.752617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 734/1452] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:28.726923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e2ebd-62a5-4b69-853c-72bfc13352ef None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:28.727497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e2ebd-62a5-4b69-853c-72bfc13352ef None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:28.727497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e2ebd-62a5-4b69-853c-72bfc13352ef None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:28.729211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:28.736290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611380b-bc20-4c74-a08a-be90f4da5510 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:28.736748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611380b-bc20-4c74-a08a-be90f4da5510 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:28.736843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611380b-bc20-4c74-a08a-be90f4da5510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:28.799689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:28.802838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0611380b-bc20-4c74-a08a-be90f4da5510 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:28.804670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0611380b-bc20-4c74-a08a-be90f4da5510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:28.900620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:29.187114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e2ebd-62a5-4b69-853c-72bfc13352ef None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:29.189010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d23e2ebd-62a5-4b69-853c-72bfc13352ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:29.292666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e2ebd-62a5-4b69-853c-72bfc13352ef None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:29.342286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:29.370056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125d685-feaf-4cdd-bd74-ee865c65d173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:29.371006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125d685-feaf-4cdd-bd74-ee865c65d173 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:29.371436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125d685-feaf-4cdd-bd74-ee865c65d173 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:29.371875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125d685-feaf-4cdd-bd74-ee865c65d173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:29.374755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2125d685-feaf-4cdd-bd74-ee865c65d173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:29.482788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125d685-feaf-4cdd-bd74-ee865c65d173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:29.484100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125d685-feaf-4cdd-bd74-ee865c65d173 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:29.501661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 720/1455] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:31.533482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce08af-8060-4bba-978b-2f4046ee2c2b None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:31.533993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce08af-8060-4bba-978b-2f4046ee2c2b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:31.533993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce08af-8060-4bba-978b-2f4046ee2c2b None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:31.535357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:31.540527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d314303-8522-43b9-b592-9395ca2d378d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:31.540921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d314303-8522-43b9-b592-9395ca2d378d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:31.541068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d314303-8522-43b9-b592-9395ca2d378d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:31.610865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5d314303-8522-43b9-b592-9395ca2d378d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:31.612074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5d314303-8522-43b9-b592-9395ca2d378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:31.642096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:31.710338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:31.999448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce08af-8060-4bba-978b-2f4046ee2c2b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:32.000994 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-58ce08af-8060-4bba-978b-2f4046ee2c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:32.103151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce08af-8060-4bba-978b-2f4046ee2c2b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:32.133423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:32.177950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651bf57-a70b-49da-950f-65bb8e84650e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:32.178548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651bf57-a70b-49da-950f-65bb8e84650e service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:32.178822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651bf57-a70b-49da-950f-65bb8e84650e service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:32.179104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651bf57-a70b-49da-950f-65bb8e84650e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:32.180934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0651bf57-a70b-49da-950f-65bb8e84650e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:32.290797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651bf57-a70b-49da-950f-65bb8e84650e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:32.292796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651bf57-a70b-49da-950f-65bb8e84650e service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:32.312545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 737/1458] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:34.262196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e810f08-acca-42da-91dc-9e10b92fbac8 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.262674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e810f08-acca-42da-91dc-9e10b92fbac8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.262779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e810f08-acca-42da-91dc-9e10b92fbac8 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.264086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:34.269445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91393e5d-68ee-4865-95c0-cd0f6e2f3641 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.269887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91393e5d-68ee-4865-95c0-cd0f6e2f3641 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.269995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91393e5d-68ee-4865-95c0-cd0f6e2f3641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.336263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-91393e5d-68ee-4865-95c0-cd0f6e2f3641 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:34.337507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-91393e5d-68ee-4865-95c0-cd0f6e2f3641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:34.347141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:34.434941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:34.716710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e810f08-acca-42da-91dc-9e10b92fbac8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.718279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0e810f08-acca-42da-91dc-9e10b92fbac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:34.822196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e810f08-acca-42da-91dc-9e10b92fbac8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.879365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:34.897801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38bf1a-bce1-4cfd-bf80-eff2a837b3b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.898398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38bf1a-bce1-4cfd-bf80-eff2a837b3b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.898671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38bf1a-bce1-4cfd-bf80-eff2a837b3b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.898932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38bf1a-bce1-4cfd-bf80-eff2a837b3b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.900517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ee38bf1a-bce1-4cfd-bf80-eff2a837b3b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:35.009000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38bf1a-bce1-4cfd-bf80-eff2a837b3b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:35.010400 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38bf1a-bce1-4cfd-bf80-eff2a837b3b0 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:35.027893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 723/1461] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:36.997459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63825b-cd9b-4ba3-bd32-92c6f79b49fb None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.998006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63825b-cd9b-4ba3-bd32-92c6f79b49fb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.998006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63825b-cd9b-4ba3-bd32-92c6f79b49fb None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.999445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:37.004731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c94ff-1b0e-427a-b82f-49f21582f94f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:37.005167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c94ff-1b0e-427a-b82f-49f21582f94f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:37.005318 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c94ff-1b0e-427a-b82f-49f21582f94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:37.069528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6f7c94ff-1b0e-427a-b82f-49f21582f94f None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:37.070720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6f7c94ff-1b0e-427a-b82f-49f21582f94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:37.109903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:37.167655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:37.449469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63825b-cd9b-4ba3-bd32-92c6f79b49fb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:37.451032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6e63825b-cd9b-4ba3-bd32-92c6f79b49fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:37.554264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63825b-cd9b-4ba3-bd32-92c6f79b49fb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:37.591490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:37.627068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acec23-3b3a-4444-a0b3-23da99b0fe00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:37.627664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acec23-3b3a-4444-a0b3-23da99b0fe00 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:37.627935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acec23-3b3a-4444-a0b3-23da99b0fe00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:37.628188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acec23-3b3a-4444-a0b3-23da99b0fe00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:37.629797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-49acec23-3b3a-4444-a0b3-23da99b0fe00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:37.736215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acec23-3b3a-4444-a0b3-23da99b0fe00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:37.737691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acec23-3b3a-4444-a0b3-23da99b0fe00 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:37.755599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 740/1464] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:39.738313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ce3c3-2fab-42c9-982a-02ce934d5627 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:39.738902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ce3c3-2fab-42c9-982a-02ce934d5627 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:39.738902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ce3c3-2fab-42c9-982a-02ce934d5627 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:39.740213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:39.746244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869068d-7516-42a9-ba41-270a1295a757 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:39.746689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869068d-7516-42a9-ba41-270a1295a757 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:39.746799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869068d-7516-42a9-ba41-270a1295a757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:39.814567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a869068d-7516-42a9-ba41-270a1295a757 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:39.815784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a869068d-7516-42a9-ba41-270a1295a757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:39.834620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:39.919764 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:40.203832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ce3c3-2fab-42c9-982a-02ce934d5627 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:40.205399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b47ce3c3-2fab-42c9-982a-02ce934d5627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:40.314266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ce3c3-2fab-42c9-982a-02ce934d5627 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:40.380560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:40.388374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7c2a0-a778-4b1e-940b-dc70ee57b1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:40.388961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7c2a0-a778-4b1e-940b-dc70ee57b1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:40.389264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7c2a0-a778-4b1e-940b-dc70ee57b1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:40.389528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7c2a0-a778-4b1e-940b-dc70ee57b1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:40.391107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d6e7c2a0-a778-4b1e-940b-dc70ee57b1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:40.498836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7c2a0-a778-4b1e-940b-dc70ee57b1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:40.500180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7c2a0-a778-4b1e-940b-dc70ee57b1c0 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:40.518484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 726/1467] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:42.459970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d935-8209-4477-ae07-ff84fa7a341d None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:42.460439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d935-8209-4477-ae07-ff84fa7a341d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:42.460542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d935-8209-4477-ae07-ff84fa7a341d None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:42.461924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:42.467908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ca77f-d155-4b2b-85e6-0f6f5b9b8d43 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:42.468363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ca77f-d155-4b2b-85e6-0f6f5b9b8d43 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:42.468524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ca77f-d155-4b2b-85e6-0f6f5b9b8d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:42.533684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-df0ca77f-d155-4b2b-85e6-0f6f5b9b8d43 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:42.534787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-df0ca77f-d155-4b2b-85e6-0f6f5b9b8d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:42.582829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:42.631285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:42.912875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d935-8209-4477-ae07-ff84fa7a341d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:42.914471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3900d935-8209-4477-ae07-ff84fa7a341d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:43.024744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d935-8209-4477-ae07-ff84fa7a341d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:43.071940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:43.098964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45917947-2b37-433c-b93e-9e856f4cf301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:43.099564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45917947-2b37-433c-b93e-9e856f4cf301 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:43.099841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45917947-2b37-433c-b93e-9e856f4cf301 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:43.100106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45917947-2b37-433c-b93e-9e856f4cf301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:43.101736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-45917947-2b37-433c-b93e-9e856f4cf301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:43.208577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45917947-2b37-433c-b93e-9e856f4cf301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:43.209949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45917947-2b37-433c-b93e-9e856f4cf301 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:43.228074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 743/1470] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:52.629555 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=22 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:53.225873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=66 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:02.993916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3c3c8-b74f-4ad8-b7d7-9951deb7ebd9 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:02.995021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3c3c8-b74f-4ad8-b7d7-9951deb7ebd9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:02.995149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3c3c8-b74f-4ad8-b7d7-9951deb7ebd9 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:02.996476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 728/1471] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:03.001894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33f9d3-b264-4655-af51-d72894cf0165 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:03.002281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33f9d3-b264-4655-af51-d72894cf0165 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:03.002437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33f9d3-b264-4655-af51-d72894cf0165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:03.070769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5b33f9d3-b264-4655-af51-d72894cf0165 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:03.071972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5b33f9d3-b264-4655-af51-d72894cf0165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:03.169883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 744/1472] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:03.471440 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc0a52-9cea-4522-b90b-c8f69af23ab3 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:03.471755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc0a52-9cea-4522-b90b-c8f69af23ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:03.472152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc0a52-9cea-4522-b90b-c8f69af23ab3 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:03.473545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 729/1473] 10.222.0.78 () {54 vars in 967 bytes} [Tue Aug 30 00:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:03.478676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba557b6-47a1-434a-8f55-4ed52106d340 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:03.479680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba557b6-47a1-434a-8f55-4ed52106d340 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:03.480333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba557b6-47a1-434a-8f55-4ed52106d340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:03.554694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4ba557b6-47a1-434a-8f55-4ed52106d340 None None] MFA Rules not processed for user `77a47e77cc1942618a3584862042c5f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:03.556082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4ba557b6-47a1-434a-8f55-4ed52106d340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:03.577158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:03.635590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 745/1474] 10.222.0.78 () {58 vars in 1064 bytes} [Tue Aug 30 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:03.643515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc0a52-9cea-4522-b90b-c8f69af23ab3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:03.645045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a3dc0a52-9cea-4522-b90b-c8f69af23ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:03.749782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc0a52-9cea-4522-b90b-c8f69af23ab3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:03.821537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af601c7e-1978-415f-97eb-c435c7e78ee5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a47e77cc1942618a3584862042c5f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:03.822053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af601c7e-1978-415f-97eb-c435c7e78ee5 service glance] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:03.822329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af601c7e-1978-415f-97eb-c435c7e78ee5 service glance] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:03.822639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af601c7e-1978-415f-97eb-c435c7e78ee5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:03.824224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-af601c7e-1978-415f-97eb-c435c7e78ee5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:03.878590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:03.933165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af601c7e-1978-415f-97eb-c435c7e78ee5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:03.934573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af601c7e-1978-415f-97eb-c435c7e78ee5 service glance] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:03.952113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 730/1475] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:05.938174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90fc54-6654-4aeb-a791-1f3ae2f55204 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:05.938766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90fc54-6654-4aeb-a791-1f3ae2f55204 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:05.938766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90fc54-6654-4aeb-a791-1f3ae2f55204 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:05.940050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 746/1476] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:05.945239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fd2fc-be7b-41e8-bb46-d81fba1e0a4b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:05.945614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fd2fc-be7b-41e8-bb46-d81fba1e0a4b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:05.945811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fd2fc-be7b-41e8-bb46-d81fba1e0a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:06.011187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-303fd2fc-be7b-41e8-bb46-d81fba1e0a4b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:06.012389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-303fd2fc-be7b-41e8-bb46-d81fba1e0a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:06.108882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 731/1477] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:06.390760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90fc54-6654-4aeb-a791-1f3ae2f55204 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:06.392291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5a90fc54-6654-4aeb-a791-1f3ae2f55204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:06.515574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90fc54-6654-4aeb-a791-1f3ae2f55204 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:06.566781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:06.613361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65d565-1bbd-4a2f-ac6c-b8b5dbe21d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:06.613963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65d565-1bbd-4a2f-ac6c-b8b5dbe21d32 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:06.614244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65d565-1bbd-4a2f-ac6c-b8b5dbe21d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:06.614500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65d565-1bbd-4a2f-ac6c-b8b5dbe21d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:06.615136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65d565-1bbd-4a2f-ac6c-b8b5dbe21d32 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:06.616840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-8b65d565-1bbd-4a2f-ac6c-b8b5dbe21d32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:56:06.616840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Aug 30 00:56:06.616840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: Aug 30 00:56:06.618003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65d565-1bbd-4a2f-ac6c-b8b5dbe21d32 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:06.638953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 747/1478] 10.222.0.78 () {56 vars in 1108 bytes} [Tue Aug 30 00:56:06 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:08.576816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273e41e-e4c7-49da-9a1a-4a87e53a9a00 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:08.577398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273e41e-e4c7-49da-9a1a-4a87e53a9a00 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:08.577398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273e41e-e4c7-49da-9a1a-4a87e53a9a00 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:08.578677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 732/1479] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:08.583793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d4981-b678-4500-8770-e06f0f0a7da7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:08.584226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d4981-b678-4500-8770-e06f0f0a7da7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:08.584334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d4981-b678-4500-8770-e06f0f0a7da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:08.653454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ea6d4981-b678-4500-8770-e06f0f0a7da7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:08.654670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ea6d4981-b678-4500-8770-e06f0f0a7da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:08.750963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 748/1480] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:09.051045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273e41e-e4c7-49da-9a1a-4a87e53a9a00 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:09.052630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7273e41e-e4c7-49da-9a1a-4a87e53a9a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:09.062731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:09.197164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273e41e-e4c7-49da-9a1a-4a87e53a9a00 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:09.279068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:09.312529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470d071-f5da-4436-a0fe-887263788f52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:09.313234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470d071-f5da-4436-a0fe-887263788f52 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:09.313660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470d071-f5da-4436-a0fe-887263788f52 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:09.314029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470d071-f5da-4436-a0fe-887263788f52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:09.316289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b470d071-f5da-4436-a0fe-887263788f52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:09.410192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470d071-f5da-4436-a0fe-887263788f52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:09.411750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470d071-f5da-4436-a0fe-887263788f52 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:09.429286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 733/1481] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:11.522820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce634cf-4ef8-4f7f-ac49-27d877379688 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:11.523348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce634cf-4ef8-4f7f-ac49-27d877379688 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:11.523348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce634cf-4ef8-4f7f-ac49-27d877379688 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:11.524612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 749/1482] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:56:11.530339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43f014-d590-4b56-8614-9192d8f87472 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:11.530753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43f014-d590-4b56-8614-9192d8f87472 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:11.530886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43f014-d590-4b56-8614-9192d8f87472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:11.597882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ed43f014-d590-4b56-8614-9192d8f87472 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:11.599093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ed43f014-d590-4b56-8614-9192d8f87472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:11.645535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:11.698337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 734/1483] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:11.990670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce634cf-4ef8-4f7f-ac49-27d877379688 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:11.992223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3ce634cf-4ef8-4f7f-ac49-27d877379688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:12.016529 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:12.099068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce634cf-4ef8-4f7f-ac49-27d877379688 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:12.171249 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e5927-bd32-4d8c-9cb7-5811e161e622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:12.171849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e5927-bd32-4d8c-9cb7-5811e161e622 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:12.172122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e5927-bd32-4d8c-9cb7-5811e161e622 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:12.172384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e5927-bd32-4d8c-9cb7-5811e161e622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:12.174007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-471e5927-bd32-4d8c-9cb7-5811e161e622 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:12.249917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:12.286374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e5927-bd32-4d8c-9cb7-5811e161e622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:12.287809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e5927-bd32-4d8c-9cb7-5811e161e622 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:12.306478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 750/1484] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:14.279281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae28ab-31c1-4c1d-92f9-f569c0788648 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.279862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae28ab-31c1-4c1d-92f9-f569c0788648 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.279862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae28ab-31c1-4c1d-92f9-f569c0788648 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.281218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 735/1485] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:14.287172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59131-fa21-4de9-9641-7cd15270b7ea None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.287631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59131-fa21-4de9-9641-7cd15270b7ea None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.287786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59131-fa21-4de9-9641-7cd15270b7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.354948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-76c59131-fa21-4de9-9641-7cd15270b7ea None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:14.356143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-76c59131-fa21-4de9-9641-7cd15270b7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:14.449899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 751/1486] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:14.749434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae28ab-31c1-4c1d-92f9-f569c0788648 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:14.751010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-53ae28ab-31c1-4c1d-92f9-f569c0788648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:14.854853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae28ab-31c1-4c1d-92f9-f569c0788648 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:14.927751 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:14.929523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3b69d-3214-4fa4-93b2-4db566d1b7e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:14.930577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3b69d-3214-4fa4-93b2-4db566d1b7e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.930967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3b69d-3214-4fa4-93b2-4db566d1b7e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.931390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3b69d-3214-4fa4-93b2-4db566d1b7e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.933578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cae3b69d-3214-4fa4-93b2-4db566d1b7e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:15.040118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3b69d-3214-4fa4-93b2-4db566d1b7e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.041513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3b69d-3214-4fa4-93b2-4db566d1b7e2 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.060286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 736/1487] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:17.133760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1095d3-60e5-4048-b0dd-1f950e3c9e53 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.134355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1095d3-60e5-4048-b0dd-1f950e3c9e53 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.134355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1095d3-60e5-4048-b0dd-1f950e3c9e53 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.136253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 752/1488] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:56:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:17.141809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0134-4465-408c-b352-3a6724b71fec None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.142244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0134-4465-408c-b352-3a6724b71fec None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.142352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0134-4465-408c-b352-3a6724b71fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.209582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-adef0134-4465-408c-b352-3a6724b71fec None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:17.210800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-adef0134-4465-408c-b352-3a6724b71fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:17.264285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:17.308692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 737/1489] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:17.602563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1095d3-60e5-4048-b0dd-1f950e3c9e53 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.604165 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0a1095d3-60e5-4048-b0dd-1f950e3c9e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:17.640031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:17.710136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1095d3-60e5-4048-b0dd-1f950e3c9e53 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.783070 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c90cc2-589a-412d-a328-02ab212fc3a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.783670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c90cc2-589a-412d-a328-02ab212fc3a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.783950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c90cc2-589a-412d-a328-02ab212fc3a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.784224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c90cc2-589a-412d-a328-02ab212fc3a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.785855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-93c90cc2-589a-412d-a328-02ab212fc3a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:17.864673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:17.894223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c90cc2-589a-412d-a328-02ab212fc3a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.895646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c90cc2-589a-412d-a328-02ab212fc3a3 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.914545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 753/1490] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:20.074516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5373dc2-2339-43d6-bf3a-94053b11396b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.075037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5373dc2-2339-43d6-bf3a-94053b11396b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.075037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5373dc2-2339-43d6-bf3a-94053b11396b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.076442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 738/1491] 10.222.0.78 () {54 vars in 927 bytes} [Tue Aug 30 00:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:20.081697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3264957-495a-41ad-8de0-f50aa00b490e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.082127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3264957-495a-41ad-8de0-f50aa00b490e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.082233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3264957-495a-41ad-8de0-f50aa00b490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.148509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a3264957-495a-41ad-8de0-f50aa00b490e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:20.149752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a3264957-495a-41ad-8de0-f50aa00b490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.246536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 754/1492] 10.222.0.78 () {58 vars in 1024 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:20.537573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5373dc2-2339-43d6-bf3a-94053b11396b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.539125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f5373dc2-2339-43d6-bf3a-94053b11396b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.640906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5373dc2-2339-43d6-bf3a-94053b11396b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.717334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ed76c-2105-4080-92df-cc86fd8b39be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.717984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ed76c-2105-4080-92df-cc86fd8b39be service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.718252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ed76c-2105-4080-92df-cc86fd8b39be service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.718520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ed76c-2105-4080-92df-cc86fd8b39be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.720118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-012ed76c-2105-4080-92df-cc86fd8b39be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.729709 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:20.828406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ed76c-2105-4080-92df-cc86fd8b39be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.829771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ed76c-2105-4080-92df-cc86fd8b39be service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.847623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 739/1493] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:30.244308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=82 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:30.845435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=50 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:33.552797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20689fb2-d952-4661-b6b2-fb69ac05eeb4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:33.553392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20689fb2-d952-4661-b6b2-fb69ac05eeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:33.553392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20689fb2-d952-4661-b6b2-fb69ac05eeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:33.628539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-20689fb2-d952-4661-b6b2-fb69ac05eeb4 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:33.629781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-20689fb2-d952-4661-b6b2-fb69ac05eeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:33.727279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 755/1494] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 00:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:33.735355 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf0309-eb10-4694-a1ee-ffe9f88119fd None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:33.735858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf0309-eb10-4694-a1ee-ffe9f88119fd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:33.735958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf0309-eb10-4694-a1ee-ffe9f88119fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:33.812582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-29bf0309-eb10-4694-a1ee-ffe9f88119fd None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:33.813734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-29bf0309-eb10-4694-a1ee-ffe9f88119fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:33.909477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 740/1495] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 00:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:33.914552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20689fb2-d952-4661-b6b2-fb69ac05eeb4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:33.915774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-20689fb2-d952-4661-b6b2-fb69ac05eeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:33.927032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20689fb2-d952-4661-b6b2-fb69ac05eeb4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:33.989183 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:34.025263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.025869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.026143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.026414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.027047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:57:34.027345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.028351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:57:34.028351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Aug 30 00:57:34.028351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: Aug 30 00:57:34.029264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.040763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 756/1496] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 00:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:57:34.048935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf0309-eb10-4694-a1ee-ffe9f88119fd None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:34.058903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf0309-eb10-4694-a1ee-ffe9f88119fd None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:34.120816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:34.154210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.154799 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.155071 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.155335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.155958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.157685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:57:34.157685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Aug 30 00:57:34.157685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: Aug 30 00:57:34.158855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.215672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 741/1497] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 00:57:34 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:34.219895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:34.221478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:34.255702 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb19aa-b3b8-433b-8be3-08374bce3331 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:34.350918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.352042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.352289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.352555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.353216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.354223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:57:34.354223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Aug 30 00:57:34.354223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: Aug 30 00:57:34.355468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.396991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 757/1498] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 00:57:34 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:34.401306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:34.402846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:34.464772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a383f-1750-4343-91f9-1e413b5e299f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:34.546750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:34.558785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.559386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.559662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.559930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.560567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.561603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:57:34.561603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Aug 30 00:57:34.561603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: Aug 30 00:57:34.562489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.575068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 742/1499] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:34.579159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:34.580665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:34.592859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090607d-aa46-4332-88c7-39074d7e2cfd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:34.618838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:34.689649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bf76b-ef21-49bd-9281-5d91c3cc7072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.690171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bf76b-ef21-49bd-9281-5d91c3cc7072 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.690448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bf76b-ef21-49bd-9281-5d91c3cc7072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.690716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bf76b-ef21-49bd-9281-5d91c3cc7072 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.691346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bf76b-ef21-49bd-9281-5d91c3cc7072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.692349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-700bf76b-ef21-49bd-9281-5d91c3cc7072 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:57:34.692349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Aug 30 00:57:34.692349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: Aug 30 00:57:34.693266 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bf76b-ef21-49bd-9281-5d91c3cc7072 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.705826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 758/1500] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:34.710032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:34.712112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:34.745617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858e136-64dd-467d-a2df-1a0ac57ed8f1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:34.842523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da2ee8-0873-4b4c-a948-dddb548c34d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.843178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da2ee8-0873-4b4c-a948-dddb548c34d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.843455 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da2ee8-0873-4b4c-a948-dddb548c34d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.843726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da2ee8-0873-4b4c-a948-dddb548c34d9 admin admin] PATH_INFO: `/v3/projects/8d491e0212194eec997227521f9e2874/users/df27a6a73e52431ea5ddb248ba87939b/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.866927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da2ee8-0873-4b4c-a948-dddb548c34d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d491e0212194eec997227521f9e2874, user_id=df27a6a73e52431ea5ddb248ba87939b, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.867970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-b3da2ee8-0873-4b4c-a948-dddb548c34d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:57:34.867970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Aug 30 00:57:34.867970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: Aug 30 00:57:34.870037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da2ee8-0873-4b4c-a948-dddb548c34d9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.894309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 743/1501] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:34 2022] PUT /identity/v3/projects/8d491e0212194eec997227521f9e2874/users/df27a6a73e52431ea5ddb248ba87939b/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:34.906996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fb77b-0623-42d7-a1f5-0ef18eccbb3a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.907291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fb77b-0623-42d7-a1f5-0ef18eccbb3a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.907566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fb77b-0623-42d7-a1f5-0ef18eccbb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.965564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3a7fb77b-0623-42d7-a1f5-0ef18eccbb3a None None] MFA Rules not processed for user `df27a6a73e52431ea5ddb248ba87939b`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:34.966705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3a7fb77b-0623-42d7-a1f5-0ef18eccbb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:35.042912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 759/1502] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 00:57:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:35.056282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857f6d1-6bda-4607-ba28-d3afec9bfcc5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:35.056565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857f6d1-6bda-4607-ba28-d3afec9bfcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:35.056832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857f6d1-6bda-4607-ba28-d3afec9bfcc5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:35.058197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 744/1503] 10.222.0.78 () {54 vars in 964 bytes} [Tue Aug 30 00:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:57:35.062883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edeeccf-9eef-47e8-a046-9d7b96e192ea None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:35.063176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edeeccf-9eef-47e8-a046-9d7b96e192ea None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:35.063453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edeeccf-9eef-47e8-a046-9d7b96e192ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:35.095684 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:35.138834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9edeeccf-9eef-47e8-a046-9d7b96e192ea None None] MFA Rules not processed for user `97e64ee3c8ee48c2ad8f351fb2a78bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:35.140038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9edeeccf-9eef-47e8-a046-9d7b96e192ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:35.216118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 760/1504] 10.222.0.78 () {58 vars in 1061 bytes} [Tue Aug 30 00:57:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:35.223781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857f6d1-6bda-4607-ba28-d3afec9bfcc5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:35.225329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0857f6d1-6bda-4607-ba28-d3afec9bfcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:35.276371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:35.332728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857f6d1-6bda-4607-ba28-d3afec9bfcc5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:35.412901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:35.413701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:35.414107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:35.414382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:35.415965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:35.516725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:35.518112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:35.521597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:35.536162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 745/1505] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:35.583400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc440f-777a-47f3-b277-d94a675cdea8 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:35.583806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc440f-777a-47f3-b277-d94a675cdea8 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:35.583968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc440f-777a-47f3-b277-d94a675cdea8 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:35.585256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 761/1506] 10.222.0.78 () {52 vars in 830 bytes} [Tue Aug 30 00:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:57:35.589026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:35.590271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:35.688632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de01d-ef45-41a7-841d-28e6a7e7f0bd service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:35.782767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:35.783423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:35.783581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:35.783852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] PATH_INFO: `/v3/users/df27a6a73e52431ea5ddb248ba87939b` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:35.797144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df27a6a73e52431ea5ddb248ba87939b)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:35.798211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:57:35.798211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Aug 30 00:57:35.798211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: Aug 30 00:57:35.799408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:35.809822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:35.867699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df27a6a73e52431ea5ddb248ba87939b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:35.876506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df27a6a73e52431ea5ddb248ba87939b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:35.877015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df27a6a73e52431ea5ddb248ba87939b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:35.877458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df27a6a73e52431ea5ddb248ba87939b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:35.900148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df27a6a73e52431ea5ddb248ba87939b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:35.919694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 746/1507] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:35 2022] DELETE /identity/v3/users/df27a6a73e52431ea5ddb248ba87939b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:35.924047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed24bf-cba6-47c3-a827-0c03bbcac50c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:35.924326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed24bf-cba6-47c3-a827-0c03bbcac50c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:35.924593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed24bf-cba6-47c3-a827-0c03bbcac50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:35.982232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b3ed24bf-cba6-47c3-a827-0c03bbcac50c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:35.983417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b3ed24bf-cba6-47c3-a827-0c03bbcac50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:36.057547 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:36.070530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 762/1508] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 00:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:36.080055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:36.081315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:36.181550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e4248-f55f-46f8-8bc1-3c20daafcc38 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:36.235330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:36.257233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59096151-0c4f-4f6c-becf-deeeb3dd21fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:36.258846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59096151-0c4f-4f6c-becf-deeeb3dd21fa service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:36.258846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59096151-0c4f-4f6c-becf-deeeb3dd21fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:36.259096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59096151-0c4f-4f6c-becf-deeeb3dd21fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:36.260687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-59096151-0c4f-4f6c-becf-deeeb3dd21fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:36.370358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59096151-0c4f-4f6c-becf-deeeb3dd21fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:36.371672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59096151-0c4f-4f6c-becf-deeeb3dd21fa service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:36.389544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 747/1509] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:36.710877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed24bf-cba6-47c3-a827-0c03bbcac50c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:36.712435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b3ed24bf-cba6-47c3-a827-0c03bbcac50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:36.723984 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed24bf-cba6-47c3-a827-0c03bbcac50c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:36.824557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:36.825215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:36.825493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:36.825761 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] PATH_INFO: `/v3/projects/8d491e0212194eec997227521f9e2874` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:36.835965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d491e0212194eec997227521f9e2874)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:36.836944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:57:36.836944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Aug 30 00:57:36.836944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: Aug 30 00:57:36.838175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:36.894323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d491e0212194eec997227521f9e2874', 'request_id': 'req-4dc13ba1-cf27-4001-b065-8d0d3c64225a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:36.907436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d491e0212194eec997227521f9e2874', 'request_id': 'req-4dc13ba1-cf27-4001-b065-8d0d3c64225a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:36.917559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d491e0212194eec997227521f9e2874', 'request_id': 'req-4dc13ba1-cf27-4001-b065-8d0d3c64225a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:36.939590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] The token cache is being invalidate because project 8d491e0212194eec997227521f9e2874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:36.939872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:36.940382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4dc13ba1-cf27-4001-b065-8d0d3c64225a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:36.941704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 763/1510] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:36 2022] DELETE /identity/v3/projects/8d491e0212194eec997227521f9e2874 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:46.387306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=62 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:46.937717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:59:07.347752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7d9a5-fad4-470d-a438-c027793f2ab8 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:07.348326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7d9a5-fad4-470d-a438-c027793f2ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:07.348326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7d9a5-fad4-470d-a438-c027793f2ab8 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:07.349852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:59:07.354938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9c09d-e1e7-4ecd-9452-9e166e4f08c0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:07.355354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9c09d-e1e7-4ecd-9452-9e166e4f08c0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:07.355488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9c09d-e1e7-4ecd-9452-9e166e4f08c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:07.437233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6fb9c09d-e1e7-4ecd-9452-9e166e4f08c0 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:07.438605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6fb9c09d-e1e7-4ecd-9452-9e166e4f08c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:07.543868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:07.702500 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7d9a5-fad4-470d-a438-c027793f2ab8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:07.704623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-99f7d9a5-fad4-470d-a438-c027793f2ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:07.828561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7d9a5-fad4-470d-a438-c027793f2ab8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:07.834820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e2fd6-647f-4941-acb1-a888572ceeac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:07.835415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e2fd6-647f-4941-acb1-a888572ceeac service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:07.835695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e2fd6-647f-4941-acb1-a888572ceeac service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:07.835966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e2fd6-647f-4941-acb1-a888572ceeac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:07.843985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e2fd6-647f-4941-acb1-a888572ceeac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:07.845573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e2fd6-647f-4941-acb1-a888572ceeac service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:07.855839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 749/1513] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:08.694806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a332bb7-3a74-45e5-9b70-228d58edc23c None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:08.696892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a332bb7-3a74-45e5-9b70-228d58edc23c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:08.696892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a332bb7-3a74-45e5-9b70-228d58edc23c None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:08.696892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:59:08.701810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783886a-dbe1-4eb3-a44b-5de94be37871 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:08.702249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783886a-dbe1-4eb3-a44b-5de94be37871 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:08.702360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783886a-dbe1-4eb3-a44b-5de94be37871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:08.722431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3783886a-dbe1-4eb3-a44b-5de94be37871 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:08.723628 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3783886a-dbe1-4eb3-a44b-5de94be37871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:08.731912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:08.891456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a332bb7-3a74-45e5-9b70-228d58edc23c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:08.906238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a332bb7-3a74-45e5-9b70-228d58edc23c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:08.912447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6a144-fc1a-4187-92f1-c03e9ad7b61e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:08.913045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6a144-fc1a-4187-92f1-c03e9ad7b61e service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:08.914417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6a144-fc1a-4187-92f1-c03e9ad7b61e service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:08.914726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6a144-fc1a-4187-92f1-c03e9ad7b61e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:08.923897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6a144-fc1a-4187-92f1-c03e9ad7b61e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:08.925536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6a144-fc1a-4187-92f1-c03e9ad7b61e service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:08.941563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 766/1516] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:10.351565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66b64e-ad21-4356-a39f-2c368526754e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:10.352104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66b64e-ad21-4356-a39f-2c368526754e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:10.352104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66b64e-ad21-4356-a39f-2c368526754e None None] PATH_INFO: `/v3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:10.353405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 751/1517] 10.222.0.47 () {54 vars in 925 bytes} [Tue Aug 30 00:59:10 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:59:10.376254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b736c-caa4-4572-bf1b-0201d5605633 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:10.376695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b736c-caa4-4572-bf1b-0201d5605633 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:10.376802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b736c-caa4-4572-bf1b-0201d5605633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:10.437058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-498b736c-caa4-4572-bf1b-0201d5605633 None None] MFA Rules not processed for user `8f860912e1f0406cadd70da440b2ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:10.438284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-498b736c-caa4-4572-bf1b-0201d5605633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:10.454967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:59:10.510791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 767/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Tue Aug 30 00:59:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:14.297615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34333b-be51-4775-bf1d-6150762e7ca1 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:14.298183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34333b-be51-4775-bf1d-6150762e7ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:14.298383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34333b-be51-4775-bf1d-6150762e7ca1 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:14.299757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 752/1519] 10.222.0.78 () {54 vars in 961 bytes} [Tue Aug 30 00:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:59:14.305781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12755f-f48d-4560-b280-bb878a8ff7ac None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:14.306233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12755f-f48d-4560-b280-bb878a8ff7ac None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:14.306340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12755f-f48d-4560-b280-bb878a8ff7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:14.326014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-bb12755f-f48d-4560-b280-bb878a8ff7ac None None] MFA Rules not processed for user `8f860912e1f0406cadd70da440b2ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:14.327222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bb12755f-f48d-4560-b280-bb878a8ff7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:14.336531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 768/1520] 10.222.0.78 () {58 vars in 1058 bytes} [Tue Aug 30 00:59:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:14.343917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34333b-be51-4775-bf1d-6150762e7ca1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:14.357639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34333b-be51-4775-bf1d-6150762e7ca1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:14.362454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87711b5-74f6-4ae7-aa47-08dd8f11b6f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:14.363055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87711b5-74f6-4ae7-aa47-08dd8f11b6f5 service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:14.363332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87711b5-74f6-4ae7-aa47-08dd8f11b6f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:14.363596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87711b5-74f6-4ae7-aa47-08dd8f11b6f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:14.371862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87711b5-74f6-4ae7-aa47-08dd8f11b6f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:14.373448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87711b5-74f6-4ae7-aa47-08dd8f11b6f5 service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:14.382991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 753/1521] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:20.140655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10504e55-267a-4eed-81e8-75101e489eef None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:20.141258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10504e55-267a-4eed-81e8-75101e489eef None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:20.141258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10504e55-267a-4eed-81e8-75101e489eef None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:20.142664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:59:20.148148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa3bcd-a0bc-4116-a5f6-0caf724609f5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:20.148597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa3bcd-a0bc-4116-a5f6-0caf724609f5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:20.148731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa3bcd-a0bc-4116-a5f6-0caf724609f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:20.169679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-95fa3bcd-a0bc-4116-a5f6-0caf724609f5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:20.170893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-95fa3bcd-a0bc-4116-a5f6-0caf724609f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:20.179693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:20.337621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10504e55-267a-4eed-81e8-75101e489eef None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:20.352256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10504e55-267a-4eed-81e8-75101e489eef None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:20.357536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405b9a5-936c-4266-981f-fea46727de8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:20.358112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405b9a5-936c-4266-981f-fea46727de8a service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:20.358386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405b9a5-936c-4266-981f-fea46727de8a service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:20.358651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405b9a5-936c-4266-981f-fea46727de8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:20.366548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405b9a5-936c-4266-981f-fea46727de8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:20.368112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405b9a5-936c-4266-981f-fea46727de8a service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:20.377400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 770/1524] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:21.214184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485d3ff-a966-4675-a295-ef4322959ca5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:21.214783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485d3ff-a966-4675-a295-ef4322959ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:21.214783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485d3ff-a966-4675-a295-ef4322959ca5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:21.216062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:59:21.220836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab420a94-23d8-4402-b032-8c7425f86330 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:21.221338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab420a94-23d8-4402-b032-8c7425f86330 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:21.221447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab420a94-23d8-4402-b032-8c7425f86330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:21.240720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ab420a94-23d8-4402-b032-8c7425f86330 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:21.242025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ab420a94-23d8-4402-b032-8c7425f86330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:21.250201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:21.405224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485d3ff-a966-4675-a295-ef4322959ca5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:21.420566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485d3ff-a966-4675-a295-ef4322959ca5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:21.426325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7432fb-ed34-4b07-9e1c-28571d5ae2a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:21.426911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7432fb-ed34-4b07-9e1c-28571d5ae2a7 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:21.427186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7432fb-ed34-4b07-9e1c-28571d5ae2a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:21.428064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7432fb-ed34-4b07-9e1c-28571d5ae2a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:21.435958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7432fb-ed34-4b07-9e1c-28571d5ae2a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:21.437566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7432fb-ed34-4b07-9e1c-28571d5ae2a7 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:21.446845 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 756/1527] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:25.071397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27ea18-2a7e-4112-afd8-dee76797a150 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:25.071906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27ea18-2a7e-4112-afd8-dee76797a150 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:25.071906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27ea18-2a7e-4112-afd8-dee76797a150 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:25.073294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:59:25.078330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33fdc7-8804-4b7c-a04a-0f45e5badd3a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:25.078759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33fdc7-8804-4b7c-a04a-0f45e5badd3a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:25.078891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33fdc7-8804-4b7c-a04a-0f45e5badd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:25.099669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2f33fdc7-8804-4b7c-a04a-0f45e5badd3a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:25.100884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2f33fdc7-8804-4b7c-a04a-0f45e5badd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:25.108797 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:59:25.406110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27ea18-2a7e-4112-afd8-dee76797a150 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:25.407790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6d27ea18-2a7e-4112-afd8-dee76797a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:25.488045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27ea18-2a7e-4112-afd8-dee76797a150 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:25.491910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf3d13-671f-4f9e-9cbf-0c682794e693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:25.492512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf3d13-671f-4f9e-9cbf-0c682794e693 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:25.492835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf3d13-671f-4f9e-9cbf-0c682794e693 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:25.493133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf3d13-671f-4f9e-9cbf-0c682794e693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:25.501860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf3d13-671f-4f9e-9cbf-0c682794e693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:25.503279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf3d13-671f-4f9e-9cbf-0c682794e693 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:25.512985 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 773/1530] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:27.397529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29197-24f1-4019-bcf9-0e5a2b1ad0b3 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:27.398120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29197-24f1-4019-bcf9-0e5a2b1ad0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:27.398120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29197-24f1-4019-bcf9-0e5a2b1ad0b3 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:27.399430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:59:27.404949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c42a55-0fca-4ab8-925f-67d8db2c9a5c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:27.405408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c42a55-0fca-4ab8-925f-67d8db2c9a5c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:27.405518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c42a55-0fca-4ab8-925f-67d8db2c9a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:27.425360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f1c42a55-0fca-4ab8-925f-67d8db2c9a5c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:27.426796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f1c42a55-0fca-4ab8-925f-67d8db2c9a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:27.435190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:27.721818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff29197-24f1-4019-bcf9-0e5a2b1ad0b3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:27.733207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff29197-24f1-4019-bcf9-0e5a2b1ad0b3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:27.736689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37fc7a-7407-4d76-9c89-55e932821e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:27.737319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37fc7a-7407-4d76-9c89-55e932821e9d service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:27.737593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37fc7a-7407-4d76-9c89-55e932821e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:27.737861 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37fc7a-7407-4d76-9c89-55e932821e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:27.747392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37fc7a-7407-4d76-9c89-55e932821e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:27.748807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37fc7a-7407-4d76-9c89-55e932821e9d service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:27.758787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 759/1533] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:31.238138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36abe9-b7f6-4eba-baa3-0fdc856b1b42 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:31.238710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36abe9-b7f6-4eba-baa3-0fdc856b1b42 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:31.238710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36abe9-b7f6-4eba-baa3-0fdc856b1b42 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:31.240651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:59:31.246047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a901df2-bf49-4a88-ba90-d8b190252280 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:31.246505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a901df2-bf49-4a88-ba90-d8b190252280 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:31.246621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a901df2-bf49-4a88-ba90-d8b190252280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:31.266493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5a901df2-bf49-4a88-ba90-d8b190252280 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:31.267729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5a901df2-bf49-4a88-ba90-d8b190252280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:31.276262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:31.831833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36abe9-b7f6-4eba-baa3-0fdc856b1b42 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:31.833630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7d36abe9-b7f6-4eba-baa3-0fdc856b1b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:31.899982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:59:31.914122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36abe9-b7f6-4eba-baa3-0fdc856b1b42 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:31.917830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bec180-0fe9-4ea2-bcaa-25c1ebd72257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:31.918436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bec180-0fe9-4ea2-bcaa-25c1ebd72257 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:31.918716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bec180-0fe9-4ea2-bcaa-25c1ebd72257 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:31.918990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bec180-0fe9-4ea2-bcaa-25c1ebd72257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:31.929173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bec180-0fe9-4ea2-bcaa-25c1ebd72257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:31.930615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bec180-0fe9-4ea2-bcaa-25c1ebd72257 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:31.940033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 776/1536] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:33.914784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a040a72-e0e2-4c83-8ce0-5c1e6ba7ef4f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:33.915355 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a040a72-e0e2-4c83-8ce0-5c1e6ba7ef4f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:33.915355 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a040a72-e0e2-4c83-8ce0-5c1e6ba7ef4f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:33.916721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:59:33.921899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd3ed4-35f5-40c5-a1d3-faf1b1042bf4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:33.922346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd3ed4-35f5-40c5-a1d3-faf1b1042bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:33.922463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd3ed4-35f5-40c5-a1d3-faf1b1042bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:33.943782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9ebd3ed4-35f5-40c5-a1d3-faf1b1042bf4 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:33.944979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9ebd3ed4-35f5-40c5-a1d3-faf1b1042bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:33.952760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:34.501926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a040a72-e0e2-4c83-8ce0-5c1e6ba7ef4f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:34.503600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6a040a72-e0e2-4c83-8ce0-5c1e6ba7ef4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:34.518193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a040a72-e0e2-4c83-8ce0-5c1e6ba7ef4f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:34.522375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333b66f-aff4-46f7-a19b-5fd132c58a9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:34.522960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333b66f-aff4-46f7-a19b-5fd132c58a9c service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:34.523231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333b66f-aff4-46f7-a19b-5fd132c58a9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:34.523494 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333b66f-aff4-46f7-a19b-5fd132c58a9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:34.532698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333b66f-aff4-46f7-a19b-5fd132c58a9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:34.534250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333b66f-aff4-46f7-a19b-5fd132c58a9c service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:34.544326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 762/1539] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:43.937607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=11 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:59:44.540096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:00:58.838510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1f2ed-0317-44d2-b941-c0ed619905c6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:58.839210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1f2ed-0317-44d2-b941-c0ed619905c6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:58.839210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1f2ed-0317-44d2-b941-c0ed619905c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:58.864941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-66e1f2ed-0317-44d2-b941-c0ed619905c6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:58.866191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-66e1f2ed-0317-44d2-b941-c0ed619905c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:58.879021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 778/1540] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:58.888008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280495-b03a-4705-b69e-d38d230c5f10 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:58.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280495-b03a-4705-b69e-d38d230c5f10 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:58.888670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280495-b03a-4705-b69e-d38d230c5f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:58.914224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-21280495-b03a-4705-b69e-d38d230c5f10 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:58.915446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-21280495-b03a-4705-b69e-d38d230c5f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:58.931416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 763/1541] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:58.936834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1f2ed-0317-44d2-b941-c0ed619905c6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:58.948470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1f2ed-0317-44d2-b941-c0ed619905c6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:58.958254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ef55e-9bf8-41d0-a8b8-878ddd32ab46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:58.958888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ef55e-9bf8-41d0-a8b8-878ddd32ab46 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:58.959164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ef55e-9bf8-41d0-a8b8-878ddd32ab46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:58.959435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ef55e-9bf8-41d0-a8b8-878ddd32ab46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:58.960092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ef55e-9bf8-41d0-a8b8-878ddd32ab46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:00:58.960393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ef55e-9bf8-41d0-a8b8-878ddd32ab46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:58.962028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ef55e-9bf8-41d0-a8b8-878ddd32ab46 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:58.986540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 779/1542] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:00:58.989875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d79df0-0daa-444f-bd0a-f0ed1b487d01 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:58.990166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d79df0-0daa-444f-bd0a-f0ed1b487d01 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:58.990442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d79df0-0daa-444f-bd0a-f0ed1b487d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:58.996710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ef55e-9bf8-41d0-a8b8-878ddd32ab46 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.010106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ef55e-9bf8-41d0-a8b8-878ddd32ab46 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.023112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadbf89-a1a7-4950-b0ee-d52d5446b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.023358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c3d79df0-0daa-444f-bd0a-f0ed1b487d01 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.023738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadbf89-a1a7-4950-b0ee-d52d5446b1cf admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.024017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadbf89-a1a7-4950-b0ee-d52d5446b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.024285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadbf89-a1a7-4950-b0ee-d52d5446b1cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.024784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c3d79df0-0daa-444f-bd0a-f0ed1b487d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.025373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadbf89-a1a7-4950-b0ee-d52d5446b1cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.026417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-9eadbf89-a1a7-4950-b0ee-d52d5446b1cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:00:59.026417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Aug 30 01:00:59.026417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: Aug 30 01:00:59.027636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadbf89-a1a7-4950-b0ee-d52d5446b1cf admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.040041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 764/1543] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.051130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a67d2-6a49-4800-84ec-3ae31cc052d5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.051471 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a67d2-6a49-4800-84ec-3ae31cc052d5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.051703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a67d2-6a49-4800-84ec-3ae31cc052d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.091084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2c9a67d2-6a49-4800-84ec-3ae31cc052d5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.092327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2c9a67d2-6a49-4800-84ec-3ae31cc052d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.101945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 765/1544] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:00:59.106994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a67d2-6a49-4800-84ec-3ae31cc052d5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.107993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 780/1545] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:00:58 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:00:59.113367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadbf89-a1a7-4950-b0ee-d52d5446b1cf admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.130009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a67d2-6a49-4800-84ec-3ae31cc052d5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.243264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadbf89-a1a7-4950-b0ee-d52d5446b1cf admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.256340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1015dbd-5d3c-4801-a3a3-b201c6bbd9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.256963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1015dbd-5d3c-4801-a3a3-b201c6bbd9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.260805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1015dbd-5d3c-4801-a3a3-b201c6bbd9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.260805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1015dbd-5d3c-4801-a3a3-b201c6bbd9f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.260805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1015dbd-5d3c-4801-a3a3-b201c6bbd9f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.261126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1015dbd-5d3c-4801-a3a3-b201c6bbd9f5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.301965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 781/1546] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:00:59.303617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a12006-a76f-4ca6-aba5-61bf07a38628 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.303939 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a12006-a76f-4ca6-aba5-61bf07a38628 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.304256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a12006-a76f-4ca6-aba5-61bf07a38628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.309418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c8868-e861-4600-a743-8bbce1eab045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.310165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8868-e861-4600-a743-8bbce1eab045 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.310451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8868-e861-4600-a743-8bbce1eab045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.311084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8868-e861-4600-a743-8bbce1eab045 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.312174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c8868-e861-4600-a743-8bbce1eab045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:00:59.312488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c8868-e861-4600-a743-8bbce1eab045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.313854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-b73c8868-e861-4600-a743-8bbce1eab045 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:00:59.313854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Aug 30 01:00:59.313854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: Aug 30 01:00:59.315853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c8868-e861-4600-a743-8bbce1eab045 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.329364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-19a12006-a76f-4ca6-aba5-61bf07a38628 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.331108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 766/1547] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:00:59.332697 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820565cf-de5c-4e05-a7f5-eb2c5ef848a7 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.332983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820565cf-de5c-4e05-a7f5-eb2c5ef848a7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.333286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820565cf-de5c-4e05-a7f5-eb2c5ef848a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.334484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-19a12006-a76f-4ca6-aba5-61bf07a38628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.344931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 782/1548] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.347111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571d18b-0da8-4b06-ae8a-98a6a69ea0af None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.347395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571d18b-0da8-4b06-ae8a-98a6a69ea0af None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.347666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571d18b-0da8-4b06-ae8a-98a6a69ea0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.358459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-820565cf-de5c-4e05-a7f5-eb2c5ef848a7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.359895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-820565cf-de5c-4e05-a7f5-eb2c5ef848a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.371146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f571d18b-0da8-4b06-ae8a-98a6a69ea0af None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.372348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f571d18b-0da8-4b06-ae8a-98a6a69ea0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.375480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 767/1549] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.377010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f593aab-dba6-4f62-bf2e-898bd0922522 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.377515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f593aab-dba6-4f62-bf2e-898bd0922522 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.377981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f593aab-dba6-4f62-bf2e-898bd0922522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.382340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 783/1550] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.383386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571d18b-0da8-4b06-ae8a-98a6a69ea0af None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.395579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571d18b-0da8-4b06-ae8a-98a6a69ea0af None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.400370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74496f-9979-4842-a533-5cd0d6908cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.400969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74496f-9979-4842-a533-5cd0d6908cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.401275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74496f-9979-4842-a533-5cd0d6908cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.401544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74496f-9979-4842-a533-5cd0d6908cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.402111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6f593aab-dba6-4f62-bf2e-898bd0922522 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.402235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74496f-9979-4842-a533-5cd0d6908cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.403733 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6f593aab-dba6-4f62-bf2e-898bd0922522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.403880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74496f-9979-4842-a533-5cd0d6908cd2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.414892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 784/1551] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:00:59.415431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 768/1552] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.416017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74496f-9979-4842-a533-5cd0d6908cd2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.417255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83662fb6-3269-4f9b-9980-5eba1d774ab4 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.418233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83662fb6-3269-4f9b-9980-5eba1d774ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.418514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83662fb6-3269-4f9b-9980-5eba1d774ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.436802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74496f-9979-4842-a533-5cd0d6908cd2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.441308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-83662fb6-3269-4f9b-9980-5eba1d774ab4 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.442216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1e491-4bee-456d-8772-5d8c946bb799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.442729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-83662fb6-3269-4f9b-9980-5eba1d774ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.442963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1e491-4bee-456d-8772-5d8c946bb799 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.443248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1e491-4bee-456d-8772-5d8c946bb799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.443520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1e491-4bee-456d-8772-5d8c946bb799 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.444192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1e491-4bee-456d-8772-5d8c946bb799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.446353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1e491-4bee-456d-8772-5d8c946bb799 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.450740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 769/1553] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.452679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e92113-95b0-4a0d-95b5-bf9174555b24 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.452963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e92113-95b0-4a0d-95b5-bf9174555b24 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.453346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e92113-95b0-4a0d-95b5-bf9174555b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.473412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-09e92113-95b0-4a0d-95b5-bf9174555b24 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.474614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-09e92113-95b0-4a0d-95b5-bf9174555b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.478774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 785/1554] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/projects => generated 391 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:00:59.480021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f59911-6265-487a-9f80-a49a60c61755 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.480307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f59911-6265-487a-9f80-a49a60c61755 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.480576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f59911-6265-487a-9f80-a49a60c61755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.485408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 770/1555] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.486642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225c7fb-194c-4067-b995-eb59c45b8307 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.487001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225c7fb-194c-4067-b995-eb59c45b8307 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.487274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225c7fb-194c-4067-b995-eb59c45b8307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.501123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-99f59911-6265-487a-9f80-a49a60c61755 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.503392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-99f59911-6265-487a-9f80-a49a60c61755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.507966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8225c7fb-194c-4067-b995-eb59c45b8307 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.509160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8225c7fb-194c-4067-b995-eb59c45b8307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.553519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:00:59.570774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 786/1556] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.570774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f59911-6265-487a-9f80-a49a60c61755 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.571072 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 771/1557] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.575332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004ff2d-95c5-45a4-8f3c-557cac656c3a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.575625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004ff2d-95c5-45a4-8f3c-557cac656c3a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.575917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004ff2d-95c5-45a4-8f3c-557cac656c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.578279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f59911-6265-487a-9f80-a49a60c61755 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.582995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb16703-8c41-4847-bd6e-ff3d7e710910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.583588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb16703-8c41-4847-bd6e-ff3d7e710910 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.583863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb16703-8c41-4847-bd6e-ff3d7e710910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.584131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb16703-8c41-4847-bd6e-ff3d7e710910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.584765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb16703-8c41-4847-bd6e-ff3d7e710910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.586403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb16703-8c41-4847-bd6e-ff3d7e710910 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.595187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2004ff2d-95c5-45a4-8f3c-557cac656c3a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.596823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2004ff2d-95c5-45a4-8f3c-557cac656c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.597668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 787/1558] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:00:59.598908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c77fb1-c9cc-4d67-a90a-e26890495065 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.599229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c77fb1-c9cc-4d67-a90a-e26890495065 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.599457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c77fb1-c9cc-4d67-a90a-e26890495065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.605234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 772/1559] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.606431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f663f6b-9164-4b9d-8d83-af58c2b580cc None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.606720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f663f6b-9164-4b9d-8d83-af58c2b580cc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.606999 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f663f6b-9164-4b9d-8d83-af58c2b580cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.622051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-07c77fb1-c9cc-4d67-a90a-e26890495065 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.623281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-07c77fb1-c9cc-4d67-a90a-e26890495065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.626669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5f663f6b-9164-4b9d-8d83-af58c2b580cc None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.628190 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5f663f6b-9164-4b9d-8d83-af58c2b580cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.631543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 788/1560] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.632029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c77fb1-c9cc-4d67-a90a-e26890495065 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.639862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 773/1561] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.641963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f663f6b-9164-4b9d-8d83-af58c2b580cc None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.642882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c77fb1-c9cc-4d67-a90a-e26890495065 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.648485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637490f4-64b1-4094-9d0a-5e47dc09aaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.649114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637490f4-64b1-4094-9d0a-5e47dc09aaeb admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.649390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637490f4-64b1-4094-9d0a-5e47dc09aaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.649658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637490f4-64b1-4094-9d0a-5e47dc09aaeb admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.650289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637490f4-64b1-4094-9d0a-5e47dc09aaeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.652108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f663f6b-9164-4b9d-8d83-af58c2b580cc None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.652405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637490f4-64b1-4094-9d0a-5e47dc09aaeb admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.656694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899336-430f-4570-9a36-2ef57658df49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.657310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899336-430f-4570-9a36-2ef57658df49 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.657590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899336-430f-4570-9a36-2ef57658df49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.657859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899336-430f-4570-9a36-2ef57658df49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.658524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93899336-430f-4570-9a36-2ef57658df49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:00:59.659579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93899336-430f-4570-9a36-2ef57658df49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.661203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93899336-430f-4570-9a36-2ef57658df49 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.675769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 774/1562] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:00:59.677772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 789/1563] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:00:59.678145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899336-430f-4570-9a36-2ef57658df49 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.678880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637490f4-64b1-4094-9d0a-5e47dc09aaeb admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.689407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637490f4-64b1-4094-9d0a-5e47dc09aaeb admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.689572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899336-430f-4570-9a36-2ef57658df49 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.694254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7d138-27f3-4ce1-8f45-ded3741f725e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.694416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37a4b2-b42f-4170-9b3b-aefd6db0aca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.694867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7d138-27f3-4ce1-8f45-ded3741f725e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.694989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37a4b2-b42f-4170-9b3b-aefd6db0aca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.695143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7d138-27f3-4ce1-8f45-ded3741f725e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.695283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37a4b2-b42f-4170-9b3b-aefd6db0aca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.695406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7d138-27f3-4ce1-8f45-ded3741f725e admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.695558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37a4b2-b42f-4170-9b3b-aefd6db0aca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.696034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7d138-27f3-4ce1-8f45-ded3741f725e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:00:59.696198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37a4b2-b42f-4170-9b3b-aefd6db0aca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:00:59.696328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7d138-27f3-4ce1-8f45-ded3741f725e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.696502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37a4b2-b42f-4170-9b3b-aefd6db0aca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.697941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7d138-27f3-4ce1-8f45-ded3741f725e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.698101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37a4b2-b42f-4170-9b3b-aefd6db0aca0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.710956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 775/1564] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:00:59.711113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 790/1565] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:00:59.711557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37a4b2-b42f-4170-9b3b-aefd6db0aca0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.711743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7d138-27f3-4ce1-8f45-ded3741f725e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.726135 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37a4b2-b42f-4170-9b3b-aefd6db0aca0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.726330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7d138-27f3-4ce1-8f45-ded3741f725e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.730618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c157be3-6af4-47db-b697-fb62c8e00a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.731214 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f96f7f-7cb8-462c-aa86-d48e73712d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.731372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c157be3-6af4-47db-b697-fb62c8e00a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.731511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c157be3-6af4-47db-b697-fb62c8e00a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.731776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c157be3-6af4-47db-b697-fb62c8e00a2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.731915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f96f7f-7cb8-462c-aa86-d48e73712d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.732195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f96f7f-7cb8-462c-aa86-d48e73712d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.732464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c157be3-6af4-47db-b697-fb62c8e00a2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:00:59.732574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f96f7f-7cb8-462c-aa86-d48e73712d35 admin admin] PATH_INFO: `/v3/projects/276fb73b4bb04c41ac33daed2e04337a/users/46624f0e6b22475fb447ea07754ef529/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.732763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c157be3-6af4-47db-b697-fb62c8e00a2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.734384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c157be3-6af4-47db-b697-fb62c8e00a2e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.735848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:00:59.746842 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 791/1566] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:00:59.748289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e7826-212c-40d5-849e-c3f37ccc2238 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.748575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e7826-212c-40d5-849e-c3f37ccc2238 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.748844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e7826-212c-40d5-849e-c3f37ccc2238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.751276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f96f7f-7cb8-462c-aa86-d48e73712d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=276fb73b4bb04c41ac33daed2e04337a, user_id=46624f0e6b22475fb447ea07754ef529, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.755438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f96f7f-7cb8-462c-aa86-d48e73712d35 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.769052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a67e7826-212c-40d5-849e-c3f37ccc2238 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:59.770163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 776/1567] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:00:59 2022] PUT /identity/v3/projects/276fb73b4bb04c41ac33daed2e04337a/users/46624f0e6b22475fb447ea07754ef529/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:00:59.770299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a67e7826-212c-40d5-849e-c3f37ccc2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.773946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f96f7f-7cb8-462c-aa86-d48e73712d35 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.834960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 792/1568] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:59.835890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e7826-212c-40d5-849e-c3f37ccc2238 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.842722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f96f7f-7cb8-462c-aa86-d48e73712d35 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.844796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e7826-212c-40d5-849e-c3f37ccc2238 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.847872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa43ef-92be-46df-b0da-42f19c0f16f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.848424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa43ef-92be-46df-b0da-42f19c0f16f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.848677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa43ef-92be-46df-b0da-42f19c0f16f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.848948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa43ef-92be-46df-b0da-42f19c0f16f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.849613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa43ef-92be-46df-b0da-42f19c0f16f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:00:59.849720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6d5d6-b883-4a19-80aa-d51a312894ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.849910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa43ef-92be-46df-b0da-42f19c0f16f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.850197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6d5d6-b883-4a19-80aa-d51a312894ed admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.850473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6d5d6-b883-4a19-80aa-d51a312894ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.850741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6d5d6-b883-4a19-80aa-d51a312894ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.851368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6d5d6-b883-4a19-80aa-d51a312894ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.851479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa43ef-92be-46df-b0da-42f19c0f16f9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.852944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6d5d6-b883-4a19-80aa-d51a312894ed admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.866413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 793/1569] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:00:59.866413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6d5d6-b883-4a19-80aa-d51a312894ed admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.866695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 777/1570] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:00:59.866695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa43ef-92be-46df-b0da-42f19c0f16f9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.874287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6d5d6-b883-4a19-80aa-d51a312894ed admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.876926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa43ef-92be-46df-b0da-42f19c0f16f9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.879346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78d1e4-caed-4a5a-a5df-a9ff0c04c25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.879938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78d1e4-caed-4a5a-a5df-a9ff0c04c25e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.880212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78d1e4-caed-4a5a-a5df-a9ff0c04c25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.880479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78d1e4-caed-4a5a-a5df-a9ff0c04c25e admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.881117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78d1e4-caed-4a5a-a5df-a9ff0c04c25e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.882553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6354b-f557-44ad-9bcf-4c321d58cacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.883005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78d1e4-caed-4a5a-a5df-a9ff0c04c25e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.883318 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6354b-f557-44ad-9bcf-4c321d58cacd admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.883596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6354b-f557-44ad-9bcf-4c321d58cacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.883979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6354b-f557-44ad-9bcf-4c321d58cacd admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.884604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6354b-f557-44ad-9bcf-4c321d58cacd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.886611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6354b-f557-44ad-9bcf-4c321d58cacd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.910448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 794/1571] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:00:59.914145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78d1e4-caed-4a5a-a5df-a9ff0c04c25e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.914425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 778/1572] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:00:59.915289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6354b-f557-44ad-9bcf-4c321d58cacd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.978262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78d1e4-caed-4a5a-a5df-a9ff0c04c25e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.980115 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6354b-f557-44ad-9bcf-4c321d58cacd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.982863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca93dc6-0a45-4c36-9f13-a49588238b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.983466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca93dc6-0a45-4c36-9f13-a49588238b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.983776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca93dc6-0a45-4c36-9f13-a49588238b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.984005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca93dc6-0a45-4c36-9f13-a49588238b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.984636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca93dc6-0a45-4c36-9f13-a49588238b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.984920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80773d0-6dda-4f17-bbf6-4d6c038b4023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.985551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80773d0-6dda-4f17-bbf6-4d6c038b4023 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.985824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80773d0-6dda-4f17-bbf6-4d6c038b4023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.986092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80773d0-6dda-4f17-bbf6-4d6c038b4023 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.986570 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca93dc6-0a45-4c36-9f13-a49588238b68 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.987289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80773d0-6dda-4f17-bbf6-4d6c038b4023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.989318 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80773d0-6dda-4f17-bbf6-4d6c038b4023 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.992250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:00.016670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 795/1573] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.019363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 779/1574] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:00:59 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.022552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80773d0-6dda-4f17-bbf6-4d6c038b4023 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.022662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762a68d-c631-4075-b2d5-5c3e0ae2dd3f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.022662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762a68d-c631-4075-b2d5-5c3e0ae2dd3f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.022858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762a68d-c631-4075-b2d5-5c3e0ae2dd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.050941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c762a68d-c631-4075-b2d5-5c3e0ae2dd3f None None] MFA Rules not processed for user `46624f0e6b22475fb447ea07754ef529`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:00.052166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c762a68d-c631-4075-b2d5-5c3e0ae2dd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:00.064441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:00.082830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80773d0-6dda-4f17-bbf6-4d6c038b4023 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.087462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74479f8-61b7-446a-a671-2c9a65a59717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.088064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74479f8-61b7-446a-a671-2c9a65a59717 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.088346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74479f8-61b7-446a-a671-2c9a65a59717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.088675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74479f8-61b7-446a-a671-2c9a65a59717 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.089359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74479f8-61b7-446a-a671-2c9a65a59717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:00.089750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74479f8-61b7-446a-a671-2c9a65a59717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.091260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74479f8-61b7-446a-a671-2c9a65a59717 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.104335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 780/1575] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:00.105265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74479f8-61b7-446a-a671-2c9a65a59717 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.121788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74479f8-61b7-446a-a671-2c9a65a59717 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.126448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f848725-0849-4aa4-b9c8-d43c4e42249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.127050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f848725-0849-4aa4-b9c8-d43c4e42249e admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.127952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f848725-0849-4aa4-b9c8-d43c4e42249e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.128215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 796/1576] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:00.128348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f848725-0849-4aa4-b9c8-d43c4e42249e admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.129055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762a68d-c631-4075-b2d5-5c3e0ae2dd3f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.130044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f848725-0849-4aa4-b9c8-d43c4e42249e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.131724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f848725-0849-4aa4-b9c8-d43c4e42249e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.139084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762a68d-c631-4075-b2d5-5c3e0ae2dd3f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.143539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 781/1577] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.147303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e6fb9-36f3-4606-8840-e6f19eb9fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.147303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e6fb9-36f3-4606-8840-e6f19eb9fea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.148214 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f848725-0849-4aa4-b9c8-d43c4e42249e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.148316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e6fb9-36f3-4606-8840-e6f19eb9fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.148316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e6fb9-36f3-4606-8840-e6f19eb9fea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.148316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e6fb9-36f3-4606-8840-e6f19eb9fea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.148316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e6fb9-36f3-4606-8840-e6f19eb9fea6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.159271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f848725-0849-4aa4-b9c8-d43c4e42249e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.164892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78021e-5304-43c8-ae46-0574a3c2328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.166111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78021e-5304-43c8-ae46-0574a3c2328f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.166248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78021e-5304-43c8-ae46-0574a3c2328f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.166524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78021e-5304-43c8-ae46-0574a3c2328f admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.167714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78021e-5304-43c8-ae46-0574a3c2328f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.169129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-dc78021e-5304-43c8-ae46-0574a3c2328f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:01:00.169129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Aug 30 01:01:00.169129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: Aug 30 01:01:00.170737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78021e-5304-43c8-ae46-0574a3c2328f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.183782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 797/1578] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.183782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e6fb9-36f3-4606-8840-e6f19eb9fea6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.195119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 782/1579] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/users => generated 287 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.196178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78021e-5304-43c8-ae46-0574a3c2328f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.206233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78021e-5304-43c8-ae46-0574a3c2328f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.250898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e6fb9-36f3-4606-8840-e6f19eb9fea6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.259253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f8c14-fcf6-405b-868d-dffd439cb40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.259738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f8c14-fcf6-405b-868d-dffd439cb40d admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.260017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f8c14-fcf6-405b-868d-dffd439cb40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.260289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f8c14-fcf6-405b-868d-dffd439cb40d admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.261060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f8c14-fcf6-405b-868d-dffd439cb40d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.263117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f8c14-fcf6-405b-868d-dffd439cb40d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.267344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ace59-1b33-4510-a183-9cf3fca14f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.267985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ace59-1b33-4510-a183-9cf3fca14f2c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.268301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ace59-1b33-4510-a183-9cf3fca14f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.268639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ace59-1b33-4510-a183-9cf3fca14f2c admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.269296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ace59-1b33-4510-a183-9cf3fca14f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.271195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ace59-1b33-4510-a183-9cf3fca14f2c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.287528 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 798/1580] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.288532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f8c14-fcf6-405b-868d-dffd439cb40d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.293013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 783/1581] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.293849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ace59-1b33-4510-a183-9cf3fca14f2c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.298798 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f8c14-fcf6-405b-868d-dffd439cb40d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.304382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca4dfa-43b3-4be0-ac0e-e0484727692d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.304974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca4dfa-43b3-4be0-ac0e-e0484727692d admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.305273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca4dfa-43b3-4be0-ac0e-e0484727692d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.305541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca4dfa-43b3-4be0-ac0e-e0484727692d admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.305662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ace59-1b33-4510-a183-9cf3fca14f2c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.306163 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca4dfa-43b3-4be0-ac0e-e0484727692d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.308093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca4dfa-43b3-4be0-ac0e-e0484727692d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.310221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6b59d-c472-4af9-a791-a19c7f479d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.310805 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6b59d-c472-4af9-a791-a19c7f479d4f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.311090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6b59d-c472-4af9-a791-a19c7f479d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.311364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6b59d-c472-4af9-a791-a19c7f479d4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.311985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6b59d-c472-4af9-a791-a19c7f479d4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.313901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6b59d-c472-4af9-a791-a19c7f479d4f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.330206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 799/1582] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.334116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca4dfa-43b3-4be0-ac0e-e0484727692d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.342818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 784/1583] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.344222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca4dfa-43b3-4be0-ac0e-e0484727692d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.344364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2239ab-0b7a-4eda-9242-0ed0f74ab6f9 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.344546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2239ab-0b7a-4eda-9242-0ed0f74ab6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.344796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2239ab-0b7a-4eda-9242-0ed0f74ab6f9 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.346731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 785/1584] 10.222.0.78 () {54 vars in 962 bytes} [Tue Aug 30 01:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:00.347455 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2239ab-0b7a-4eda-9242-0ed0f74ab6f9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.371186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:00.408339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ee809-a6dc-4a0c-82b6-2d04b91ba252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.409739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ee809-a6dc-4a0c-82b6-2d04b91ba252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.410016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ee809-a6dc-4a0c-82b6-2d04b91ba252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.410286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ee809-a6dc-4a0c-82b6-2d04b91ba252 admin admin] PATH_INFO: `/v3/projects/ceebbaa5dd9d4110a6393190c5b466f0/users/5c95c8ce27fd43c6a6fe10443b6bde13/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.413733 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2239ab-0b7a-4eda-9242-0ed0f74ab6f9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.418550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccc9af-b7af-4e9e-907e-cb38afe19016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.419246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccc9af-b7af-4e9e-907e-cb38afe19016 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.419587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccc9af-b7af-4e9e-907e-cb38afe19016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.419932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccc9af-b7af-4e9e-907e-cb38afe19016 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.421640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eccc9af-b7af-4e9e-907e-cb38afe19016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.422686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eccc9af-b7af-4e9e-907e-cb38afe19016 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.430366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ee809-a6dc-4a0c-82b6-2d04b91ba252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceebbaa5dd9d4110a6393190c5b466f0, user_id=5c95c8ce27fd43c6a6fe10443b6bde13, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.431442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-983ee809-a6dc-4a0c-82b6-2d04b91ba252 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:01:00.431442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Aug 30 01:01:00.431442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: Aug 30 01:01:00.433424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ee809-a6dc-4a0c-82b6-2d04b91ba252 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.446534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 786/1585] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.447028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 800/1586] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:00 2022] PUT /identity/v3/projects/ceebbaa5dd9d4110a6393190c5b466f0/users/5c95c8ce27fd43c6a6fe10443b6bde13/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:00.450803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ee809-a6dc-4a0c-82b6-2d04b91ba252 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.450914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccc9af-b7af-4e9e-907e-cb38afe19016 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.456482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:00.460722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccc9af-b7af-4e9e-907e-cb38afe19016 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.512685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ee809-a6dc-4a0c-82b6-2d04b91ba252 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.517030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a11b1d-e14e-4e6f-a870-b68ad2cacfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.517255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6412d30-2089-458d-bdf0-289f92597f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.517773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a11b1d-e14e-4e6f-a870-b68ad2cacfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.518505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6412d30-2089-458d-bdf0-289f92597f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.518714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a11b1d-e14e-4e6f-a870-b68ad2cacfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.518895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a11b1d-e14e-4e6f-a870-b68ad2cacfe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.519001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6412d30-2089-458d-bdf0-289f92597f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.519158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6412d30-2089-458d-bdf0-289f92597f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.519520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a11b1d-e14e-4e6f-a870-b68ad2cacfe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.519893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6412d30-2089-458d-bdf0-289f92597f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.521127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a11b1d-e14e-4e6f-a870-b68ad2cacfe8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.521538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6412d30-2089-458d-bdf0-289f92597f7e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.530997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 787/1587] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.531430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 801/1588] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.532076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6412d30-2089-458d-bdf0-289f92597f7e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.534725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a11b1d-e14e-4e6f-a870-b68ad2cacfe8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.542409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6412d30-2089-458d-bdf0-289f92597f7e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.543891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a11b1d-e14e-4e6f-a870-b68ad2cacfe8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.546973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77bdb5-af64-487b-846d-8ef50182429a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.547556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77bdb5-af64-487b-846d-8ef50182429a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.548091 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77bdb5-af64-487b-846d-8ef50182429a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.548362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77bdb5-af64-487b-846d-8ef50182429a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.548777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbd8a-942a-41ef-83ee-87d905645533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.548988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77bdb5-af64-487b-846d-8ef50182429a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.549418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbd8a-942a-41ef-83ee-87d905645533 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.549694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbd8a-942a-41ef-83ee-87d905645533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.550055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbd8a-942a-41ef-83ee-87d905645533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.550682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbd8a-942a-41ef-83ee-87d905645533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.550795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77bdb5-af64-487b-846d-8ef50182429a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.552229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbd8a-942a-41ef-83ee-87d905645533 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.560643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 802/1589] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.565784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 788/1590] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.565784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c02cd2-1c87-4c4a-b68e-c4c6b5a1eb9b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.565784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c02cd2-1c87-4c4a-b68e-c4c6b5a1eb9b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.565784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c02cd2-1c87-4c4a-b68e-c4c6b5a1eb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.566352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77bdb5-af64-487b-846d-8ef50182429a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.575795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77bdb5-af64-487b-846d-8ef50182429a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.581210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649e630-4516-481d-8fd2-95f6dd17a83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.581663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649e630-4516-481d-8fd2-95f6dd17a83e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.582010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649e630-4516-481d-8fd2-95f6dd17a83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.582284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649e630-4516-481d-8fd2-95f6dd17a83e admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.582913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649e630-4516-481d-8fd2-95f6dd17a83e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.584812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649e630-4516-481d-8fd2-95f6dd17a83e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.602593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f4c02cd2-1c87-4c4a-b68e-c4c6b5a1eb9b None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:00.603801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f4c02cd2-1c87-4c4a-b68e-c4c6b5a1eb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:00.610044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 803/1591] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:00.610044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649e630-4516-481d-8fd2-95f6dd17a83e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.617838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649e630-4516-481d-8fd2-95f6dd17a83e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.622390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ed439-ce95-417b-b026-87fbfc46e21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.623033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ed439-ce95-417b-b026-87fbfc46e21d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.623151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ed439-ce95-417b-b026-87fbfc46e21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.623408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ed439-ce95-417b-b026-87fbfc46e21d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.624034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ed439-ce95-417b-b026-87fbfc46e21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.625649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ed439-ce95-417b-b026-87fbfc46e21d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.637045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 804/1592] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.638382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596ce2-215e-42f8-b710-3ae9cc65ebd2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.638664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596ce2-215e-42f8-b710-3ae9cc65ebd2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.638936 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596ce2-215e-42f8-b710-3ae9cc65ebd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.657771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-40596ce2-215e-42f8-b710-3ae9cc65ebd2 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:00.659643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-40596ce2-215e-42f8-b710-3ae9cc65ebd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:00.667350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 805/1593] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:00.668102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40596ce2-215e-42f8-b710-3ae9cc65ebd2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.675944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 789/1594] 10.222.0.78 () {58 vars in 1059 bytes} [Tue Aug 30 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:00.677971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c02cd2-1c87-4c4a-b68e-c4c6b5a1eb9b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.678628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40596ce2-215e-42f8-b710-3ae9cc65ebd2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.683178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d7b80-ecee-4578-a2c7-4346b92cf632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.683770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d7b80-ecee-4578-a2c7-4346b92cf632 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.684036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d7b80-ecee-4578-a2c7-4346b92cf632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.684299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d7b80-ecee-4578-a2c7-4346b92cf632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.684935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d7b80-ecee-4578-a2c7-4346b92cf632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.686555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d7b80-ecee-4578-a2c7-4346b92cf632 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.689356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c02cd2-1c87-4c4a-b68e-c4c6b5a1eb9b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.694248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03baa-445e-43fe-973c-2027bdd4e420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.694851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03baa-445e-43fe-973c-2027bdd4e420 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.695133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03baa-445e-43fe-973c-2027bdd4e420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.695404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03baa-445e-43fe-973c-2027bdd4e420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.696045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e03baa-445e-43fe-973c-2027bdd4e420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.699109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 806/1595] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.699109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d7b80-ecee-4578-a2c7-4346b92cf632 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.699321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e03baa-445e-43fe-973c-2027bdd4e420 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.711850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 790/1596] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.712011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d7b80-ecee-4578-a2c7-4346b92cf632 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.712207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03baa-445e-43fe-973c-2027bdd4e420 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.715996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeea1c-7183-43c7-aaaa-28d16be10166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.716579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeea1c-7183-43c7-aaaa-28d16be10166 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.716853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeea1c-7183-43c7-aaaa-28d16be10166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.717142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeea1c-7183-43c7-aaaa-28d16be10166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.717776 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfeea1c-7183-43c7-aaaa-28d16be10166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.719339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfeea1c-7183-43c7-aaaa-28d16be10166 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.722142 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03baa-445e-43fe-973c-2027bdd4e420 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.726722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207763f1-b278-447c-b248-5bbe2b13ad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.727302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207763f1-b278-447c-b248-5bbe2b13ad42 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.727581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207763f1-b278-447c-b248-5bbe2b13ad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.727897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207763f1-b278-447c-b248-5bbe2b13ad42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.728610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207763f1-b278-447c-b248-5bbe2b13ad42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.730250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 807/1597] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.730446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207763f1-b278-447c-b248-5bbe2b13ad42 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.731141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeea1c-7183-43c7-aaaa-28d16be10166 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.740797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeea1c-7183-43c7-aaaa-28d16be10166 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.744515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 791/1598] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.744515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207763f1-b278-447c-b248-5bbe2b13ad42 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.749813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc84c3a-bf9d-4039-96a1-e844eb749bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.750439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc84c3a-bf9d-4039-96a1-e844eb749bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.750767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc84c3a-bf9d-4039-96a1-e844eb749bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.751035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc84c3a-bf9d-4039-96a1-e844eb749bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.751668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc84c3a-bf9d-4039-96a1-e844eb749bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.752943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207763f1-b278-447c-b248-5bbe2b13ad42 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.753379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc84c3a-bf9d-4039-96a1-e844eb749bd0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.757643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294dbe5-580a-4c5b-ace7-8c3234292772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.758216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294dbe5-580a-4c5b-ace7-8c3234292772 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.758496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294dbe5-580a-4c5b-ace7-8c3234292772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.758774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294dbe5-580a-4c5b-ace7-8c3234292772 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.759415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294dbe5-580a-4c5b-ace7-8c3234292772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.761023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294dbe5-580a-4c5b-ace7-8c3234292772 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.763955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 808/1599] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.764809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc84c3a-bf9d-4039-96a1-e844eb749bd0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.772339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 792/1600] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:00.773181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294dbe5-580a-4c5b-ace7-8c3234292772 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.785971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294dbe5-580a-4c5b-ace7-8c3234292772 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.791474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e768e-a76e-4877-adf8-c6534c786a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.792061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e768e-a76e-4877-adf8-c6534c786a0d service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.792342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e768e-a76e-4877-adf8-c6534c786a0d service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.792643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e768e-a76e-4877-adf8-c6534c786a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.795779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:00.825948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc84c3a-bf9d-4039-96a1-e844eb749bd0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.826892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:00.831034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af87c-0cc6-41fc-a4dc-e96f1561e297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.831978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af87c-0cc6-41fc-a4dc-e96f1561e297 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.832386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af87c-0cc6-41fc-a4dc-e96f1561e297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.832836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af87c-0cc6-41fc-a4dc-e96f1561e297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.842924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4af87c-0cc6-41fc-a4dc-e96f1561e297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.844343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4af87c-0cc6-41fc-a4dc-e96f1561e297 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.852866 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e768e-a76e-4877-adf8-c6534c786a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.854437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e768e-a76e-4877-adf8-c6534c786a0d service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.854783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 809/1601] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:00.855501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af87c-0cc6-41fc-a4dc-e96f1561e297 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.865558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af87c-0cc6-41fc-a4dc-e96f1561e297 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.867765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 793/1602] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:00.868422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e768e-a76e-4877-adf8-c6534c786a0d service nova] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.870746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418cb7c-87fa-423a-964c-63c63ab69d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.871699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418cb7c-87fa-423a-964c-63c63ab69d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.872072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418cb7c-87fa-423a-964c-63c63ab69d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.872487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418cb7c-87fa-423a-964c-63c63ab69d1c admin admin] PATH_INFO: `/v3/projects/f40b3e29a18b48cba6800c9f3a0dfee9/users/c6fe0e0033de447da70be9bc0228eba2/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.880034 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e768e-a76e-4877-adf8-c6534c786a0d service nova] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.894039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418cb7c-87fa-423a-964c-63c63ab69d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40b3e29a18b48cba6800c9f3a0dfee9, user_id=c6fe0e0033de447da70be9bc0228eba2, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.895853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551c3d3-eb9f-4c82-935e-4f2b8548cf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.896526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551c3d3-eb9f-4c82-935e-4f2b8548cf79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.896808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551c3d3-eb9f-4c82-935e-4f2b8548cf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.897143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551c3d3-eb9f-4c82-935e-4f2b8548cf79 admin admin] PATH_INFO: `/v3/projects/b5c84d6a896d4b6499dd668dcaad3783/users/44d911e3e6254ec29df39a460220efd2/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.901394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418cb7c-87fa-423a-964c-63c63ab69d1c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.919060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551c3d3-eb9f-4c82-935e-4f2b8548cf79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c84d6a896d4b6499dd668dcaad3783, user_id=44d911e3e6254ec29df39a460220efd2, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.921776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551c3d3-eb9f-4c82-935e-4f2b8548cf79 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.928711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 810/1603] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:00 2022] PUT /identity/v3/projects/f40b3e29a18b48cba6800c9f3a0dfee9/users/c6fe0e0033de447da70be9bc0228eba2/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:00.932652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418cb7c-87fa-423a-964c-63c63ab69d1c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.946968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 794/1604] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:00 2022] PUT /identity/v3/projects/b5c84d6a896d4b6499dd668dcaad3783/users/44d911e3e6254ec29df39a460220efd2/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:00.959395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551c3d3-eb9f-4c82-935e-4f2b8548cf79 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:01.038337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418cb7c-87fa-423a-964c-63c63ab69d1c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:01.079585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551c3d3-eb9f-4c82-935e-4f2b8548cf79 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:01.090987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f6ef7-1ae9-4ea8-bb95-c5df479cc323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:01.092381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f6ef7-1ae9-4ea8-bb95-c5df479cc323 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.092666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f6ef7-1ae9-4ea8-bb95-c5df479cc323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.092947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f6ef7-1ae9-4ea8-bb95-c5df479cc323 admin admin] PATH_INFO: `/v3/projects/0596692dbfb149139a1e72159c6f93db/users/cc7cf306aadb45d1b65252b53c86b612/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.123192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f6ef7-1ae9-4ea8-bb95-c5df479cc323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0596692dbfb149139a1e72159c6f93db, user_id=cc7cf306aadb45d1b65252b53c86b612, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:01.126156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f6ef7-1ae9-4ea8-bb95-c5df479cc323 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:01.147575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113734ea-2e52-4fa9-801e-f03fa6f59ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:01.148627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113734ea-2e52-4fa9-801e-f03fa6f59ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.149106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113734ea-2e52-4fa9-801e-f03fa6f59ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.149514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 795/1605] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:00 2022] PUT /identity/v3/projects/0596692dbfb149139a1e72159c6f93db/users/cc7cf306aadb45d1b65252b53c86b612/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:01.149846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113734ea-2e52-4fa9-801e-f03fa6f59ef4 admin admin] PATH_INFO: `/v3/projects/7728131ac0f34701b17f8508a83c1e96/users/8baaf27dca57456786ae37d6ee4557f5/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.150547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f6ef7-1ae9-4ea8-bb95-c5df479cc323 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:01.177295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113734ea-2e52-4fa9-801e-f03fa6f59ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7728131ac0f34701b17f8508a83c1e96, user_id=8baaf27dca57456786ae37d6ee4557f5, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:01.180119 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113734ea-2e52-4fa9-801e-f03fa6f59ef4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:01.194703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 811/1606] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:00 2022] PUT /identity/v3/projects/7728131ac0f34701b17f8508a83c1e96/users/8baaf27dca57456786ae37d6ee4557f5/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:01.195752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113734ea-2e52-4fa9-801e-f03fa6f59ef4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:01.254668 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f6ef7-1ae9-4ea8-bb95-c5df479cc323 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:01.288099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:01.309312 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:01.335275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111baad-cd7f-4d9b-9733-7d5d0fd09742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:01.336448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111baad-cd7f-4d9b-9733-7d5d0fd09742 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.338235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111baad-cd7f-4d9b-9733-7d5d0fd09742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.338910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111baad-cd7f-4d9b-9733-7d5d0fd09742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.339608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113734ea-2e52-4fa9-801e-f03fa6f59ef4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:01.339883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111baad-cd7f-4d9b-9733-7d5d0fd09742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:01.343029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111baad-cd7f-4d9b-9733-7d5d0fd09742 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:01.348222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbf447-df93-4c17-845a-0a75a047a246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:01.351013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbf447-df93-4c17-845a-0a75a047a246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.351745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbf447-df93-4c17-845a-0a75a047a246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.352161 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbf447-df93-4c17-845a-0a75a047a246 admin admin] PATH_INFO: `/v3/projects/0c0f91770bb14bb6920fc277936c189e/users/e4cc89946c2e45a0a75eef16cb042758/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.358273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 796/1607] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:01.359871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9feff7-2038-41ee-93aa-a5dde8f083ef None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.360419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9feff7-2038-41ee-93aa-a5dde8f083ef None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.360780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9feff7-2038-41ee-93aa-a5dde8f083ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.383986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbf447-df93-4c17-845a-0a75a047a246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0f91770bb14bb6920fc277936c189e, user_id=e4cc89946c2e45a0a75eef16cb042758, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:01.385246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1a9feff7-2038-41ee-93aa-a5dde8f083ef None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:01.386476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1a9feff7-2038-41ee-93aa-a5dde8f083ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:01.387682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbf447-df93-4c17-845a-0a75a047a246 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:01.394892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 797/1608] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:01.396229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79606159-5778-45c9-a289-c506472fac14 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.396572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79606159-5778-45c9-a289-c506472fac14 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.396860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79606159-5778-45c9-a289-c506472fac14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.411393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 812/1609] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:01 2022] PUT /identity/v3/projects/0c0f91770bb14bb6920fc277936c189e/users/e4cc89946c2e45a0a75eef16cb042758/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:01.412856 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e2efd-4183-4369-bccb-1bb038f3012a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.413162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e2efd-4183-4369-bccb-1bb038f3012a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.413478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e2efd-4183-4369-bccb-1bb038f3012a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.415269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 813/1610] 10.222.0.78 () {54 vars in 912 bytes} [Tue Aug 30 01:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:01.417961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55356d50-b362-4231-a29d-e76c33c4a296 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.418242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55356d50-b362-4231-a29d-e76c33c4a296 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.418514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55356d50-b362-4231-a29d-e76c33c4a296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.435033 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-79606159-5778-45c9-a289-c506472fac14 None None] MFA Rules not processed for user `44d911e3e6254ec29df39a460220efd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:01.436413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-79606159-5778-45c9-a289-c506472fac14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:01.439571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-55356d50-b362-4231-a29d-e76c33c4a296 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:01.441487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-55356d50-b362-4231-a29d-e76c33c4a296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:01.522023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 798/1611] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:01.523288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9dcd2-ae4f-4511-bab0-a881306315d0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.523560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9dcd2-ae4f-4511-bab0-a881306315d0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.523962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9dcd2-ae4f-4511-bab0-a881306315d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.532981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 814/1612] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:01.534612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55356d50-b362-4231-a29d-e76c33c4a296 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:01.555805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55356d50-b362-4231-a29d-e76c33c4a296 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:01.561923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd91e7-4de8-41be-bd87-c498dc2e917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:01.562660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd91e7-4de8-41be-bd87-c498dc2e917f admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.563012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd91e7-4de8-41be-bd87-c498dc2e917f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.563353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd91e7-4de8-41be-bd87-c498dc2e917f admin admin] PATH_INFO: `/v3/projects/8899fe6080f54675ae19c42c7db7ec9c/users/bd8121ee9e65453ca7c8884a61656dbc/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.572836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-dba9dcd2-ae4f-4511-bab0-a881306315d0 None None] MFA Rules not processed for user `8baaf27dca57456786ae37d6ee4557f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:01.574103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-dba9dcd2-ae4f-4511-bab0-a881306315d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:01.583484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd91e7-4de8-41be-bd87-c498dc2e917f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8899fe6080f54675ae19c42c7db7ec9c, user_id=bd8121ee9e65453ca7c8884a61656dbc, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:01.586310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd91e7-4de8-41be-bd87-c498dc2e917f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:01.599610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 815/1613] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:01 2022] PUT /identity/v3/projects/8899fe6080f54675ae19c42c7db7ec9c/users/bd8121ee9e65453ca7c8884a61656dbc/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:01.600522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd91e7-4de8-41be-bd87-c498dc2e917f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:01.654943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 799/1614] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:01.656600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26944a1c-586d-40e9-ac0e-1620f7d6579b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.656600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26944a1c-586d-40e9-ac0e-1620f7d6579b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.656815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26944a1c-586d-40e9-ac0e-1620f7d6579b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.664189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd91e7-4de8-41be-bd87-c498dc2e917f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:01.669060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013225c8-aaf0-4d41-a6d4-a164affc772e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:01.669785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013225c8-aaf0-4d41-a6d4-a164affc772e service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.670133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013225c8-aaf0-4d41-a6d4-a164affc772e service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.670610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013225c8-aaf0-4d41-a6d4-a164affc772e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.678268 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-26944a1c-586d-40e9-ac0e-1620f7d6579b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:01.679474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-26944a1c-586d-40e9-ac0e-1620f7d6579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:01.726067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:01.735137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:01.750073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013225c8-aaf0-4d41-a6d4-a164affc772e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:01.751694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013225c8-aaf0-4d41-a6d4-a164affc772e service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:01.754712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 800/1615] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:01.756184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c93c2-db6c-4174-9025-d9df8540c8e7 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.756470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c93c2-db6c-4174-9025-d9df8540c8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.756747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c93c2-db6c-4174-9025-d9df8540c8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.765518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 816/1616] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:01.767593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013225c8-aaf0-4d41-a6d4-a164affc772e service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:01.781545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8c6c93c2-db6c-4174-9025-d9df8540c8e7 None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:01.784474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8c6c93c2-db6c-4174-9025-d9df8540c8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:01.808221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013225c8-aaf0-4d41-a6d4-a164affc772e service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:01.814961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33b1e5-a401-4a38-adb6-f07049d90958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:01.817891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33b1e5-a401-4a38-adb6-f07049d90958 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.818495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33b1e5-a401-4a38-adb6-f07049d90958 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.819741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33b1e5-a401-4a38-adb6-f07049d90958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.840961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33b1e5-a401-4a38-adb6-f07049d90958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:01.845523 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33b1e5-a401-4a38-adb6-f07049d90958 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:01.864697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 817/1617] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:01.867644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a387cd-fc0c-498c-83c9-6185c4f95ed0 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.868031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a387cd-fc0c-498c-83c9-6185c4f95ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.868672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a387cd-fc0c-498c-83c9-6185c4f95ed0 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.871183 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 818/1618] 10.222.0.78 () {54 vars in 962 bytes} [Tue Aug 30 01:01:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:01.872458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742643-866d-4548-a1d1-8c36c3c55343 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.872748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742643-866d-4548-a1d1-8c36c3c55343 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.873023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742643-866d-4548-a1d1-8c36c3c55343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.901452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 801/1619] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:01.902394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c93c2-db6c-4174-9025-d9df8540c8e7 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:01.917375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c93c2-db6c-4174-9025-d9df8540c8e7 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:01.925413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffdedb-b2b3-4148-98aa-76e6237b0c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:01.926685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffdedb-b2b3-4148-98aa-76e6237b0c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:01.927182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffdedb-b2b3-4148-98aa-76e6237b0c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:01.927556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffdedb-b2b3-4148-98aa-76e6237b0c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:01.929203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffdedb-b2b3-4148-98aa-76e6237b0c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:01.931869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffdedb-b2b3-4148-98aa-76e6237b0c40 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:01.933184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-22742643-866d-4548-a1d1-8c36c3c55343 None None] MFA Rules not processed for user `bd8121ee9e65453ca7c8884a61656dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:01.935025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-22742643-866d-4548-a1d1-8c36c3c55343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:01.967107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 802/1620] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:01 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:01.968078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffdedb-b2b3-4148-98aa-76e6237b0c40 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:01.970133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0dffdedb-b2b3-4148-98aa-76e6237b0c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:02.028335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 819/1621] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:02.030398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edf637-6018-4af9-a3c9-30e269915725 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.030803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edf637-6018-4af9-a3c9-30e269915725 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.031172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edf637-6018-4af9-a3c9-30e269915725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.055409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffdedb-b2b3-4148-98aa-76e6237b0c40 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.061110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-52edf637-6018-4af9-a3c9-30e269915725 None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:02.061967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71c780-8bfb-45ce-9c29-0bac5b7af828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.062452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-52edf637-6018-4af9-a3c9-30e269915725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:02.062924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71c780-8bfb-45ce-9c29-0bac5b7af828 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.063401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71c780-8bfb-45ce-9c29-0bac5b7af828 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.063769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71c780-8bfb-45ce-9c29-0bac5b7af828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.155855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 820/1622] 10.222.0.78 () {58 vars in 1059 bytes} [Tue Aug 30 01:01:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:02.156920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edf637-6018-4af9-a3c9-30e269915725 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.157055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71c780-8bfb-45ce-9c29-0bac5b7af828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.158878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71c780-8bfb-45ce-9c29-0bac5b7af828 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.177618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 803/1623] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:02.179399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71c780-8bfb-45ce-9c29-0bac5b7af828 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.193477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71c780-8bfb-45ce-9c29-0bac5b7af828 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.206110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f02ddf-dd94-450f-aee0-860e460fa0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.207804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f02ddf-dd94-450f-aee0-860e460fa0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.208087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f02ddf-dd94-450f-aee0-860e460fa0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.208360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f02ddf-dd94-450f-aee0-860e460fa0b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.211897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f02ddf-dd94-450f-aee0-860e460fa0b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.218503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f02ddf-dd94-450f-aee0-860e460fa0b1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.243933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:02.266261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edf637-6018-4af9-a3c9-30e269915725 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.271482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 804/1624] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:02 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:02.274337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f02ddf-dd94-450f-aee0-860e460fa0b1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.281101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8faef-fb54-45ad-820a-f8c814bf6677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.283144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8faef-fb54-45ad-820a-f8c814bf6677 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.284310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8faef-fb54-45ad-820a-f8c814bf6677 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.286244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8faef-fb54-45ad-820a-f8c814bf6677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.304025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f02ddf-dd94-450f-aee0-860e460fa0b1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.306508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:02.312992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8faef-fb54-45ad-820a-f8c814bf6677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.316128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7175af-e1c5-453f-a805-e8af8aaa9b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.318680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7175af-e1c5-453f-a805-e8af8aaa9b6e admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.319174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7175af-e1c5-453f-a805-e8af8aaa9b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.319395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8faef-fb54-45ad-820a-f8c814bf6677 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.319799 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7175af-e1c5-453f-a805-e8af8aaa9b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.321926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7175af-e1c5-453f-a805-e8af8aaa9b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.328350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7175af-e1c5-453f-a805-e8af8aaa9b6e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.341200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 821/1625] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:02.341691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8faef-fb54-45ad-820a-f8c814bf6677 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.371171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8faef-fb54-45ad-820a-f8c814bf6677 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.388031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed22bd-1f22-4e9d-af3e-80582a8e1a14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.390412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed22bd-1f22-4e9d-af3e-80582a8e1a14 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.391129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed22bd-1f22-4e9d-af3e-80582a8e1a14 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.391129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed22bd-1f22-4e9d-af3e-80582a8e1a14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.399831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 805/1626] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:02 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:02.400634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7175af-e1c5-453f-a805-e8af8aaa9b6e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.561592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed22bd-1f22-4e9d-af3e-80582a8e1a14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.564473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed22bd-1f22-4e9d-af3e-80582a8e1a14 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.570651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7175af-e1c5-453f-a805-e8af8aaa9b6e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.580329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c5928-a7e1-4d26-998d-ef5bdaeda375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.584412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c5928-a7e1-4d26-998d-ef5bdaeda375 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.584698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c5928-a7e1-4d26-998d-ef5bdaeda375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.584974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c5928-a7e1-4d26-998d-ef5bdaeda375 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.585684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c5928-a7e1-4d26-998d-ef5bdaeda375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.590026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 822/1627] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:02.590927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed22bd-1f22-4e9d-af3e-80582a8e1a14 service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.592451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c5928-a7e1-4d26-998d-ef5bdaeda375 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.615941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed22bd-1f22-4e9d-af3e-80582a8e1a14 service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.619926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 806/1628] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:02.624409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c5928-a7e1-4d26-998d-ef5bdaeda375 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.637695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c5928-a7e1-4d26-998d-ef5bdaeda375 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.644983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef1a67-c493-4b80-ab28-dc81bb3bbf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.645352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80dd0f-ec34-44eb-87d0-719202976228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.645875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef1a67-c493-4b80-ab28-dc81bb3bbf29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.645989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80dd0f-ec34-44eb-87d0-719202976228 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.646182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80dd0f-ec34-44eb-87d0-719202976228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.646415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef1a67-c493-4b80-ab28-dc81bb3bbf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.646540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80dd0f-ec34-44eb-87d0-719202976228 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.646800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef1a67-c493-4b80-ab28-dc81bb3bbf29 admin admin] PATH_INFO: `/v3/projects/56487006d08942f996065d66c17e2fd7/users/947b770e17b840469f2dd86fbfe348b4/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.647073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80dd0f-ec34-44eb-87d0-719202976228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.649009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80dd0f-ec34-44eb-87d0-719202976228 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.678080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef1a67-c493-4b80-ab28-dc81bb3bbf29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56487006d08942f996065d66c17e2fd7, user_id=947b770e17b840469f2dd86fbfe348b4, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.681796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef1a67-c493-4b80-ab28-dc81bb3bbf29 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.711485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 823/1629] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:02 2022] POST /identity/v3/users => generated 308 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:02.715223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80dd0f-ec34-44eb-87d0-719202976228 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.718017 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 807/1630] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:02 2022] PUT /identity/v3/projects/56487006d08942f996065d66c17e2fd7/users/947b770e17b840469f2dd86fbfe348b4/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:02.723607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef1a67-c493-4b80-ab28-dc81bb3bbf29 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.726657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80dd0f-ec34-44eb-87d0-719202976228 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.872505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d5d48-daad-42ec-b8da-13d90e4e1b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.877970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d5d48-daad-42ec-b8da-13d90e4e1b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.878738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d5d48-daad-42ec-b8da-13d90e4e1b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.879220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d5d48-daad-42ec-b8da-13d90e4e1b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.881479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d5d48-daad-42ec-b8da-13d90e4e1b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.884899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d5d48-daad-42ec-b8da-13d90e4e1b20 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.906022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 824/1631] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:01:02.908087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d5d48-daad-42ec-b8da-13d90e4e1b20 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.918889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef1a67-c493-4b80-ab28-dc81bb3bbf29 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.920680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d5d48-daad-42ec-b8da-13d90e4e1b20 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.930602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be741e2-176a-4ac7-9d57-3275d38df342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.931340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be741e2-176a-4ac7-9d57-3275d38df342 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.931630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be741e2-176a-4ac7-9d57-3275d38df342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.931914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be741e2-176a-4ac7-9d57-3275d38df342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.932689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be741e2-176a-4ac7-9d57-3275d38df342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.934208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f5ff2-5754-4394-bb08-92e23408abea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.935226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be741e2-176a-4ac7-9d57-3275d38df342 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.937022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f5ff2-5754-4394-bb08-92e23408abea admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.943328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f5ff2-5754-4394-bb08-92e23408abea admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.945024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f5ff2-5754-4394-bb08-92e23408abea admin admin] PATH_INFO: `/v3/projects/56b9ef99b8474ec09de294cd00528c90/users/00d06a2a51884269b42d7e536f41c3b4/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:02.957541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 808/1632] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:02.959823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be741e2-176a-4ac7-9d57-3275d38df342 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:02.982111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be741e2-176a-4ac7-9d57-3275d38df342 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:02.989846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f5ff2-5754-4394-bb08-92e23408abea admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b9ef99b8474ec09de294cd00528c90, user_id=00d06a2a51884269b42d7e536f41c3b4, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:02.992959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f5ff2-5754-4394-bb08-92e23408abea admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:02.997127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce6e2b-fd2c-45cd-bbf0-ce63bea8245f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:02.997904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce6e2b-fd2c-45cd-bbf0-ce63bea8245f admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:02.998183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce6e2b-fd2c-45cd-bbf0-ce63bea8245f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:02.998503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce6e2b-fd2c-45cd-bbf0-ce63bea8245f admin admin] PATH_INFO: `/v3/projects/56487006d08942f996065d66c17e2fd7/users/947b770e17b840469f2dd86fbfe348b4/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:03.002802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce6e2b-fd2c-45cd-bbf0-ce63bea8245f admin admin] RBAC: Authorizing `identity:create_grant(project_id=56487006d08942f996065d66c17e2fd7, user_id=947b770e17b840469f2dd86fbfe348b4, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:03.006914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce6e2b-fd2c-45cd-bbf0-ce63bea8245f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:03.036524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 825/1633] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:02 2022] PUT /identity/v3/projects/56b9ef99b8474ec09de294cd00528c90/users/00d06a2a51884269b42d7e536f41c3b4/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:03.043720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f5ff2-5754-4394-bb08-92e23408abea admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:03.170608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 809/1634] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:02 2022] PUT /identity/v3/projects/56487006d08942f996065d66c17e2fd7/users/947b770e17b840469f2dd86fbfe348b4/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:03.170608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d2a5c-f64b-4cd9-860a-3210d574ccd8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:03.170608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d2a5c-f64b-4cd9-860a-3210d574ccd8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:03.170608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d2a5c-f64b-4cd9-860a-3210d574ccd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:03.182056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-527d2a5c-f64b-4cd9-860a-3210d574ccd8 None None] MFA Rules not processed for user `947b770e17b840469f2dd86fbfe348b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:03.184583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-527d2a5c-f64b-4cd9-860a-3210d574ccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:03.253521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:03.275688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:03.309567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f5ff2-5754-4394-bb08-92e23408abea admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:03.396377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 810/1635] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:03.402606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed62c8-7e29-4341-8df4-c409f764c55b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:03.403296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed62c8-7e29-4341-8df4-c409f764c55b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:03.403401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed62c8-7e29-4341-8df4-c409f764c55b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:03.405241 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 811/1636] 10.222.0.78 () {54 vars in 912 bytes} [Tue Aug 30 01:01:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:03.417992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a65b62-4c03-4d7c-9b65-0c9debcecb9e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:03.418542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a65b62-4c03-4d7c-9b65-0c9debcecb9e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:03.420366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a65b62-4c03-4d7c-9b65-0c9debcecb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:03.467667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-56a65b62-4c03-4d7c-9b65-0c9debcecb9e None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:03.469247 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-56a65b62-4c03-4d7c-9b65-0c9debcecb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:03.487399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d49cfa-fdee-412f-ab08-a8a2dea41b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:03.488188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d49cfa-fdee-412f-ab08-a8a2dea41b7a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:03.488580 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d49cfa-fdee-412f-ab08-a8a2dea41b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:03.488945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d49cfa-fdee-412f-ab08-a8a2dea41b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:03.489706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d49cfa-fdee-412f-ab08-a8a2dea41b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:03.491559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d49cfa-fdee-412f-ab08-a8a2dea41b7a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:03.518178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 826/1637] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:03.519260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d49cfa-fdee-412f-ab08-a8a2dea41b7a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:03.638454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 812/1638] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:01:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:03.639350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a65b62-4c03-4d7c-9b65-0c9debcecb9e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:03.662575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d49cfa-fdee-412f-ab08-a8a2dea41b7a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:03.666865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a65b62-4c03-4d7c-9b65-0c9debcecb9e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:03.671072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd96cb-7b53-4299-8191-23e64615e9df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:03.672381 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd96cb-7b53-4299-8191-23e64615e9df service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:03.673518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd96cb-7b53-4299-8191-23e64615e9df service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:03.675231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd96cb-7b53-4299-8191-23e64615e9df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:03.676488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2073d07-fe19-45ca-95e5-808e4a2d85e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:03.677223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2073d07-fe19-45ca-95e5-808e4a2d85e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:03.677406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2073d07-fe19-45ca-95e5-808e4a2d85e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:03.677682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2073d07-fe19-45ca-95e5-808e4a2d85e1 admin admin] PATH_INFO: `/v3/projects/56b9ef99b8474ec09de294cd00528c90/users/00d06a2a51884269b42d7e536f41c3b4/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:03.696765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd96cb-7b53-4299-8191-23e64615e9df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:03.697229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2073d07-fe19-45ca-95e5-808e4a2d85e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b9ef99b8474ec09de294cd00528c90, user_id=00d06a2a51884269b42d7e536f41c3b4, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:03.700318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd96cb-7b53-4299-8191-23e64615e9df service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:03.700826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2073d07-fe19-45ca-95e5-808e4a2d85e1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:03.719819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 827/1639] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:03.721049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd96cb-7b53-4299-8191-23e64615e9df service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:03.723892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 813/1640] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:03 2022] PUT /identity/v3/projects/56b9ef99b8474ec09de294cd00528c90/users/00d06a2a51884269b42d7e536f41c3b4/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:03.733720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd96cb-7b53-4299-8191-23e64615e9df service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:03.734224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280d028-1cae-485f-bdfd-7f7ab0acdaf1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:03.734681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280d028-1cae-485f-bdfd-7f7ab0acdaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:03.735106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280d028-1cae-485f-bdfd-7f7ab0acdaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:03.790977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4280d028-1cae-485f-bdfd-7f7ab0acdaf1 None None] MFA Rules not processed for user `00d06a2a51884269b42d7e536f41c3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:03.796207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4280d028-1cae-485f-bdfd-7f7ab0acdaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:03.847750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7ad77-0164-45cb-82bb-636b94233687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:03.850137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7ad77-0164-45cb-82bb-636b94233687 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:03.850724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7ad77-0164-45cb-82bb-636b94233687 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:03.851823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7ad77-0164-45cb-82bb-636b94233687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:03.959750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:04.002067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7ad77-0164-45cb-82bb-636b94233687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:04.004398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7ad77-0164-45cb-82bb-636b94233687 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:04.027528 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 828/1641] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:04.081909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 814/1642] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:03 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:04.203318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7ad77-0164-45cb-82bb-636b94233687 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:04.219634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7ad77-0164-45cb-82bb-636b94233687 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:04.238456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d70f6-a550-4298-b6b4-94cb9dd1a2cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:04.245573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d70f6-a550-4298-b6b4-94cb9dd1a2cd service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:04.246116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d70f6-a550-4298-b6b4-94cb9dd1a2cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:04.246771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d70f6-a550-4298-b6b4-94cb9dd1a2cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:04.287911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d70f6-a550-4298-b6b4-94cb9dd1a2cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:04.289811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d70f6-a550-4298-b6b4-94cb9dd1a2cd service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:04.317367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 829/1643] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:04 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:04.438360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280d028-1cae-485f-bdfd-7f7ab0acdaf1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:04.455484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280d028-1cae-485f-bdfd-7f7ab0acdaf1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:04.466902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d263af-a9e8-47ce-96d4-250f915e86c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:04.467589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d263af-a9e8-47ce-96d4-250f915e86c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:04.467791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d263af-a9e8-47ce-96d4-250f915e86c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:04.468060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d263af-a9e8-47ce-96d4-250f915e86c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:04.497470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:04.606008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d263af-a9e8-47ce-96d4-250f915e86c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:04.607531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d263af-a9e8-47ce-96d4-250f915e86c4 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:04.629339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 815/1644] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:05.012209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d70f6-a550-4298-b6b4-94cb9dd1a2cd service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:05.161964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d70f6-a550-4298-b6b4-94cb9dd1a2cd service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:05.171423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:05.172824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:05.173532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:05.174729 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] PATH_INFO: `/v3/users/bd8121ee9e65453ca7c8884a61656dbc` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:05.177384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8121ee9e65453ca7c8884a61656dbc)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:05.179090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: WARNING py.warnings [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:01:05.179090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: warnings.warn(msg) Aug 30 01:01:05.179090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: Aug 30 01:01:05.180833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:05.309361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8121ee9e65453ca7c8884a61656dbc'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:05.328527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8121ee9e65453ca7c8884a61656dbc'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:05.361055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8121ee9e65453ca7c8884a61656dbc'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:05.362511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8121ee9e65453ca7c8884a61656dbc'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:05.387338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8121ee9e65453ca7c8884a61656dbc'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:05.399561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 830/1645] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:05 2022] DELETE /identity/v3/users/bd8121ee9e65453ca7c8884a61656dbc => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:05.401070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5a1fe-6122-4519-b5a0-24cc955a88fa None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:05.401600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5a1fe-6122-4519-b5a0-24cc955a88fa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:05.401698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5a1fe-6122-4519-b5a0-24cc955a88fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:05.443135 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-84b5a1fe-6122-4519-b5a0-24cc955a88fa None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:05.444431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-84b5a1fe-6122-4519-b5a0-24cc955a88fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:05.586042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 816/1646] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:05.826698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:05.828143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:05.997030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:06.056819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fea2c-8001-4a46-83fb-3b67ec424bad admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:06.064404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4a907-58da-4ca7-8f3b-a250b2002039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:06.066590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4a907-58da-4ca7-8f3b-a250b2002039 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:06.067037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4a907-58da-4ca7-8f3b-a250b2002039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:06.067645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4a907-58da-4ca7-8f3b-a250b2002039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:06.088545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4a907-58da-4ca7-8f3b-a250b2002039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:06.091248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4a907-58da-4ca7-8f3b-a250b2002039 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:06.105404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 831/1647] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:08.293659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5a1fe-6122-4519-b5a0-24cc955a88fa None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:08.295745 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-84b5a1fe-6122-4519-b5a0-24cc955a88fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:08.345840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5a1fe-6122-4519-b5a0-24cc955a88fa None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:08.354961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:08.356799 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:08.357273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:08.357740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] PATH_INFO: `/v3/projects/8899fe6080f54675ae19c42c7db7ec9c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:08.380960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8899fe6080f54675ae19c42c7db7ec9c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:08.382372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:01:08.382372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Aug 30 01:01:08.382372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: Aug 30 01:01:08.384581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:08.531136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8899fe6080f54675ae19c42c7db7ec9c', 'request_id': 'req-c7bb0927-1be0-464f-9ce3-48e090a9a41e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:08.569116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8899fe6080f54675ae19c42c7db7ec9c', 'request_id': 'req-c7bb0927-1be0-464f-9ce3-48e090a9a41e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:08.587681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8899fe6080f54675ae19c42c7db7ec9c', 'request_id': 'req-c7bb0927-1be0-464f-9ce3-48e090a9a41e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:08.634569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] The token cache is being invalidate because project 8899fe6080f54675ae19c42c7db7ec9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:08.634875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:08.637533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:08.642278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 817/1648] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:08 2022] DELETE /identity/v3/projects/8899fe6080f54675ae19c42c7db7ec9c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:08.647566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4a907-58da-4ca7-8f3b-a250b2002039 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:08.663376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4a907-58da-4ca7-8f3b-a250b2002039 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:08.946702 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:08.947390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:08.947667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:08.947934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] PATH_INFO: `/v3/users/00d06a2a51884269b42d7e536f41c3b4` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:08.952029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d06a2a51884269b42d7e536f41c3b4)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:08.960458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:09.094378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d06a2a51884269b42d7e536f41c3b4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:09.114487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d06a2a51884269b42d7e536f41c3b4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:09.124090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d06a2a51884269b42d7e536f41c3b4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:09.126598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d06a2a51884269b42d7e536f41c3b4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:09.204648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfad5282-d8ac-4d0e-8686-f36ce38b958f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d06a2a51884269b42d7e536f41c3b4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:09.247227 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 832/1649] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:08 2022] DELETE /identity/v3/users/00d06a2a51884269b42d7e536f41c3b4 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:09.956977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:09.958578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:10.005311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0927-1be0-464f-9ce3-48e090a9a41e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:10.171116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:10.269348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:10.276740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:10.277172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:10.298372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] PATH_INFO: `/v3/users/44d911e3e6254ec29df39a460220efd2` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:10.301946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44d911e3e6254ec29df39a460220efd2)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:10.308506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:10.509419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44d911e3e6254ec29df39a460220efd2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:10.532337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44d911e3e6254ec29df39a460220efd2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:10.532971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44d911e3e6254ec29df39a460220efd2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:10.540763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44d911e3e6254ec29df39a460220efd2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:10.589560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44d911e3e6254ec29df39a460220efd2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:10.647654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 818/1650] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:09 2022] DELETE /identity/v3/users/44d911e3e6254ec29df39a460220efd2 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:10.649848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0772d2-ab43-468d-b5f1-42b99ac84135 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:10.650359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0772d2-ab43-468d-b5f1-42b99ac84135 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:10.650440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0772d2-ab43-468d-b5f1-42b99ac84135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:10.696687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-bd0772d2-ab43-468d-b5f1-42b99ac84135 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:10.697878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bd0772d2-ab43-468d-b5f1-42b99ac84135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:10.945127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 833/1651] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:11.039582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:11.041651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:11.167660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d6557-67be-4016-abbc-36081937c442 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:11.176042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e85fb-8e4a-47e9-ab98-eca481ff255d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:11.176915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e85fb-8e4a-47e9-ab98-eca481ff255d service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:11.177407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e85fb-8e4a-47e9-ab98-eca481ff255d service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:11.177825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e85fb-8e4a-47e9-ab98-eca481ff255d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:11.207820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e85fb-8e4a-47e9-ab98-eca481ff255d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:11.209261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e85fb-8e4a-47e9-ab98-eca481ff255d service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:11.230926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 819/1652] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:11.749635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0772d2-ab43-468d-b5f1-42b99ac84135 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:11.752352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bd0772d2-ab43-468d-b5f1-42b99ac84135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:11.762382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:11.769055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0772d2-ab43-468d-b5f1-42b99ac84135 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:11.794860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:11.795745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:11.796214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:11.796687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] PATH_INFO: `/v3/projects/56b9ef99b8474ec09de294cd00528c90` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:11.798446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b9ef99b8474ec09de294cd00528c90)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:11.801610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:11.917204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b9ef99b8474ec09de294cd00528c90', 'request_id': 'req-a5872524-e79e-4522-83e5-828e3fc20316'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:11.935418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b9ef99b8474ec09de294cd00528c90', 'request_id': 'req-a5872524-e79e-4522-83e5-828e3fc20316'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:11.947937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b9ef99b8474ec09de294cd00528c90', 'request_id': 'req-a5872524-e79e-4522-83e5-828e3fc20316'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:11.972405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] The token cache is being invalidate because project 56b9ef99b8474ec09de294cd00528c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:11.973034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:11.985672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a5872524-e79e-4522-83e5-828e3fc20316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:11.988980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 834/1653] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:11 2022] DELETE /identity/v3/projects/56b9ef99b8474ec09de294cd00528c90 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:11.993385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c7c44-1ab0-4c54-8e70-d90be93ff5c3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:11.993889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c7c44-1ab0-4c54-8e70-d90be93ff5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:11.994299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c7c44-1ab0-4c54-8e70-d90be93ff5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:12.039005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-340c7c44-1ab0-4c54-8e70-d90be93ff5c3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:12.040361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-340c7c44-1ab0-4c54-8e70-d90be93ff5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:12.170699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:12.199679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 820/1654] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:12.209358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea05c9-4785-4cf8-a875-9251ed1cd82b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:12.209641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea05c9-4785-4cf8-a875-9251ed1cd82b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:12.209913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea05c9-4785-4cf8-a875-9251ed1cd82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:12.250353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-87ea05c9-4785-4cf8-a875-9251ed1cd82b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:12.251591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-87ea05c9-4785-4cf8-a875-9251ed1cd82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:12.291895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 835/1655] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:12.296524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c7c44-1ab0-4c54-8e70-d90be93ff5c3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:12.313061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c7c44-1ab0-4c54-8e70-d90be93ff5c3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:12.337603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ed923-52dd-4c2a-a24b-2865a2f14799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:12.340605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ed923-52dd-4c2a-a24b-2865a2f14799 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:12.340605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ed923-52dd-4c2a-a24b-2865a2f14799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:12.340605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ed923-52dd-4c2a-a24b-2865a2f14799 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:12.345587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ed923-52dd-4c2a-a24b-2865a2f14799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:12.346306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ed923-52dd-4c2a-a24b-2865a2f14799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:12.349870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ed923-52dd-4c2a-a24b-2865a2f14799 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:12.377850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 821/1656] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:12.379577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea05c9-4785-4cf8-a875-9251ed1cd82b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:12.391130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea05c9-4785-4cf8-a875-9251ed1cd82b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:12.402498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50750efc-2449-4d16-96a7-b9f7dd8e3950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:12.403556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50750efc-2449-4d16-96a7-b9f7dd8e3950 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:12.403940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50750efc-2449-4d16-96a7-b9f7dd8e3950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:12.404361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50750efc-2449-4d16-96a7-b9f7dd8e3950 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:12.405149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50750efc-2449-4d16-96a7-b9f7dd8e3950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:12.408260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50750efc-2449-4d16-96a7-b9f7dd8e3950 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:12.454240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 836/1657] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:12 2022] POST /identity/v3/projects => generated 357 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:01:12.458216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ed923-52dd-4c2a-a24b-2865a2f14799 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:12.474882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ed923-52dd-4c2a-a24b-2865a2f14799 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:12.598870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280af05-f1b1-4fca-9e4b-aada26f2381a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:12.599633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280af05-f1b1-4fca-9e4b-aada26f2381a admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:12.600292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280af05-f1b1-4fca-9e4b-aada26f2381a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:12.600636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280af05-f1b1-4fca-9e4b-aada26f2381a admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:12.601486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280af05-f1b1-4fca-9e4b-aada26f2381a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:12.603811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280af05-f1b1-4fca-9e4b-aada26f2381a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:12.635969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 822/1658] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:12 2022] POST /identity/v3/users => generated 279 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:12.638767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50750efc-2449-4d16-96a7-b9f7dd8e3950 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:12.654642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50750efc-2449-4d16-96a7-b9f7dd8e3950 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:12.666036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c288-e6f9-41de-a700-bff9c3d6e0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:12.668441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c288-e6f9-41de-a700-bff9c3d6e0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:12.668805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c288-e6f9-41de-a700-bff9c3d6e0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:12.670237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c288-e6f9-41de-a700-bff9c3d6e0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:12.671088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6c288-e6f9-41de-a700-bff9c3d6e0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:12.675905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6c288-e6f9-41de-a700-bff9c3d6e0b1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:12.700792 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 837/1659] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:12.706327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280af05-f1b1-4fca-9e4b-aada26f2381a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:12.726653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280af05-f1b1-4fca-9e4b-aada26f2381a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:12.737803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8245df-c3fa-4660-915e-f46d0875e891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:12.738425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8245df-c3fa-4660-915e-f46d0875e891 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:12.738754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8245df-c3fa-4660-915e-f46d0875e891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:12.739206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8245df-c3fa-4660-915e-f46d0875e891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:12.739868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8245df-c3fa-4660-915e-f46d0875e891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:12.741553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8245df-c3fa-4660-915e-f46d0875e891 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:12.757123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 823/1660] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:12.760572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c288-e6f9-41de-a700-bff9c3d6e0b1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:12.803737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c288-e6f9-41de-a700-bff9c3d6e0b1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:12.819486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4b655-139f-4814-a406-331a95888017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:12.820286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4b655-139f-4814-a406-331a95888017 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:12.820686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4b655-139f-4814-a406-331a95888017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:12.821221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4b655-139f-4814-a406-331a95888017 admin admin] PATH_INFO: `/v3/projects/7951cc4f90214786a438cc94e80ace18/users/3f90d1ff80064806aa7b760f1fbca680/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:12.848299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4b655-139f-4814-a406-331a95888017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7951cc4f90214786a438cc94e80ace18, user_id=3f90d1ff80064806aa7b760f1fbca680, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:12.852581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4b655-139f-4814-a406-331a95888017 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:12.902666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 838/1661] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:12 2022] PUT /identity/v3/projects/7951cc4f90214786a438cc94e80ace18/users/3f90d1ff80064806aa7b760f1fbca680/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:12.904472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397a43d-176b-4d8f-a666-66f5130f4bcd None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:12.904811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397a43d-176b-4d8f-a666-66f5130f4bcd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:12.905040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397a43d-176b-4d8f-a666-66f5130f4bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:12.963322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b397a43d-176b-4d8f-a666-66f5130f4bcd None None] MFA Rules not processed for user `3f90d1ff80064806aa7b760f1fbca680`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:12.964775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b397a43d-176b-4d8f-a666-66f5130f4bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:13.035611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85cc0d-5d4d-416c-8a7a-8aee6c07b5a3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:13.037944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85cc0d-5d4d-416c-8a7a-8aee6c07b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:13.038376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85cc0d-5d4d-416c-8a7a-8aee6c07b5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:13.099283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2d85cc0d-5d4d-416c-8a7a-8aee6c07b5a3 None None] MFA Rules not processed for user `5c95c8ce27fd43c6a6fe10443b6bde13`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:13.100513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2d85cc0d-5d4d-416c-8a7a-8aee6c07b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:13.148381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 824/1662] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:13.153043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397a43d-176b-4d8f-a666-66f5130f4bcd None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:13.301207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:13.317602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397a43d-176b-4d8f-a666-66f5130f4bcd None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:13.327270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345459a-3e8b-458a-b4cd-d5692d7c50c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:13.327903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345459a-3e8b-458a-b4cd-d5692d7c50c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:13.328180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345459a-3e8b-458a-b4cd-d5692d7c50c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:13.328451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345459a-3e8b-458a-b4cd-d5692d7c50c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:13.329315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345459a-3e8b-458a-b4cd-d5692d7c50c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:13.340299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345459a-3e8b-458a-b4cd-d5692d7c50c1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:13.347446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 839/1663] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:13.351999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d74cf-2640-4f93-9c61-4eb911ca1f11 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:13.352284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d74cf-2640-4f93-9c61-4eb911ca1f11 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:13.352563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d74cf-2640-4f93-9c61-4eb911ca1f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:13.385712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-364d74cf-2640-4f93-9c61-4eb911ca1f11 None None] MFA Rules not processed for user `5c95c8ce27fd43c6a6fe10443b6bde13`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:13.388361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-364d74cf-2640-4f93-9c61-4eb911ca1f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:13.392756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 825/1664] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:13 2022] POST /identity/v3/projects => generated 355 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:13.393579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345459a-3e8b-458a-b4cd-d5692d7c50c1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:13.425605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:13.528097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 840/1665] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:13.529350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d74cf-2640-4f93-9c61-4eb911ca1f11 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:13.530950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-364d74cf-2640-4f93-9c61-4eb911ca1f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:13.622065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d74cf-2640-4f93-9c61-4eb911ca1f11 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:13.630175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ac64d-6169-4159-ae3f-6aa7ecd6d803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:13.630949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ac64d-6169-4159-ae3f-6aa7ecd6d803 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:13.631305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ac64d-6169-4159-ae3f-6aa7ecd6d803 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:13.631652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ac64d-6169-4159-ae3f-6aa7ecd6d803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:13.639980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345459a-3e8b-458a-b4cd-d5692d7c50c1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:13.641918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ac64d-6169-4159-ae3f-6aa7ecd6d803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:13.645495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ac64d-6169-4159-ae3f-6aa7ecd6d803 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:13.668658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 841/1666] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:13.670391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549b05d-3c4f-449b-a713-9e64651cce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:13.674208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549b05d-3c4f-449b-a713-9e64651cce1d admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:13.674518 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549b05d-3c4f-449b-a713-9e64651cce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:13.675582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549b05d-3c4f-449b-a713-9e64651cce1d admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:13.676259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549b05d-3c4f-449b-a713-9e64651cce1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:13.683323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549b05d-3c4f-449b-a713-9e64651cce1d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:13.781636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 826/1667] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:13 2022] POST /identity/v3/users => generated 284 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:13.782233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ac64d-6169-4159-ae3f-6aa7ecd6d803 service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:13.805489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ac64d-6169-4159-ae3f-6aa7ecd6d803 service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:13.814154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0df30-f8c5-46c3-9aba-59c7f57325ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:13.815747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0df30-f8c5-46c3-9aba-59c7f57325ef admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:13.816178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0df30-f8c5-46c3-9aba-59c7f57325ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:13.816725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0df30-f8c5-46c3-9aba-59c7f57325ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:13.817538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0df30-f8c5-46c3-9aba-59c7f57325ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:13.819436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0df30-f8c5-46c3-9aba-59c7f57325ef admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:13.838733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 842/1668] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:13.839815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549b05d-3c4f-449b-a713-9e64651cce1d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:13.864610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549b05d-3c4f-449b-a713-9e64651cce1d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:13.896022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713c823-9ced-4199-b647-2319a6424f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:13.906827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713c823-9ced-4199-b647-2319a6424f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:13.908493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713c823-9ced-4199-b647-2319a6424f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:13.912573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713c823-9ced-4199-b647-2319a6424f8f admin admin] PATH_INFO: `/v3/projects/20dbde44f7d84f87b124c48bb9d2fdaa/users/532a47038bac49b7b4264860421bd534/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:13.991735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713c823-9ced-4199-b647-2319a6424f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20dbde44f7d84f87b124c48bb9d2fdaa, user_id=532a47038bac49b7b4264860421bd534, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:14.002597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713c823-9ced-4199-b647-2319a6424f8f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:14.047501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 827/1669] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:13 2022] PUT /identity/v3/projects/20dbde44f7d84f87b124c48bb9d2fdaa/users/532a47038bac49b7b4264860421bd534/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:14.052546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0df30-f8c5-46c3-9aba-59c7f57325ef admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:14.083772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0df30-f8c5-46c3-9aba-59c7f57325ef admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:14.122935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713c823-9ced-4199-b647-2319a6424f8f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:14.133612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2713c823-9ced-4199-b647-2319a6424f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:14.250315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061f02-35ca-4b31-a1e3-d22574ce6b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:14.250950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061f02-35ca-4b31-a1e3-d22574ce6b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:14.251231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061f02-35ca-4b31-a1e3-d22574ce6b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:14.251504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061f02-35ca-4b31-a1e3-d22574ce6b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:14.252752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061f02-35ca-4b31-a1e3-d22574ce6b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:14.262778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061f02-35ca-4b31-a1e3-d22574ce6b9b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:14.335808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 843/1670] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:14.336755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061f02-35ca-4b31-a1e3-d22574ce6b9b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:14.367257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713c823-9ced-4199-b647-2319a6424f8f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:14.374388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061f02-35ca-4b31-a1e3-d22574ce6b9b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:14.383310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:14.384297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:14.384683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:14.385178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] PATH_INFO: `/v3/projects/b5c84d6a896d4b6499dd668dcaad3783` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:14.387565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5c84d6a896d4b6499dd668dcaad3783)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:14.390537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:14.429934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c700f-94e7-49d8-8988-8317f4d4f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:14.430624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c700f-94e7-49d8-8988-8317f4d4f6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:14.430906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c700f-94e7-49d8-8988-8317f4d4f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:14.431177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c700f-94e7-49d8-8988-8317f4d4f6fa admin admin] PATH_INFO: `/v3/projects/20dbde44f7d84f87b124c48bb9d2fdaa/users/532a47038bac49b7b4264860421bd534/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:14.440469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c700f-94e7-49d8-8988-8317f4d4f6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=20dbde44f7d84f87b124c48bb9d2fdaa, user_id=532a47038bac49b7b4264860421bd534, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:14.459263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c700f-94e7-49d8-8988-8317f4d4f6fa admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:14.485803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 844/1671] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:14 2022] PUT /identity/v3/projects/20dbde44f7d84f87b124c48bb9d2fdaa/users/532a47038bac49b7b4264860421bd534/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:14.488378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bdf8a-ba44-47da-8282-a78746712954 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:14.488673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bdf8a-ba44-47da-8282-a78746712954 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:14.488948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bdf8a-ba44-47da-8282-a78746712954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:14.502519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:14.510073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5c84d6a896d4b6499dd668dcaad3783', 'request_id': 'req-b11eb42e-e541-4208-8976-5754ed4734ec'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:14.530670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5c84d6a896d4b6499dd668dcaad3783', 'request_id': 'req-b11eb42e-e541-4208-8976-5754ed4734ec'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:14.544224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5c84d6a896d4b6499dd668dcaad3783', 'request_id': 'req-b11eb42e-e541-4208-8976-5754ed4734ec'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:14.548508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:14.554446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3c1bdf8a-ba44-47da-8282-a78746712954 None None] MFA Rules not processed for user `532a47038bac49b7b4264860421bd534`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:14.558868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3c1bdf8a-ba44-47da-8282-a78746712954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:14.576758 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] The token cache is being invalidate because project b5c84d6a896d4b6499dd668dcaad3783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:14.577154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:14.578691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:14.588835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 828/1672] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:14 2022] DELETE /identity/v3/projects/b5c84d6a896d4b6499dd668dcaad3783 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:14.591265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:14.596052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:14.801308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11eb42e-e541-4208-8976-5754ed4734ec admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:14.804574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 845/1673] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:14.806769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c8751-4bb6-4225-8453-2eba82104bff None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:14.807047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c8751-4bb6-4225-8453-2eba82104bff None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:14.807314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c8751-4bb6-4225-8453-2eba82104bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:14.824601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:14.826077 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:14.826360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:14.828458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] PATH_INFO: `/v3/users/46624f0e6b22475fb447ea07754ef529` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:14.836410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46624f0e6b22475fb447ea07754ef529)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:14.838862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:14.910341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e48c8751-4bb6-4225-8453-2eba82104bff None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:14.917635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e48c8751-4bb6-4225-8453-2eba82104bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:14.971639 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 846/1674] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:14.973686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c8751-4bb6-4225-8453-2eba82104bff None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:14.979882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e48c8751-4bb6-4225-8453-2eba82104bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:15.033894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46624f0e6b22475fb447ea07754ef529'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:15.066261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46624f0e6b22475fb447ea07754ef529'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:15.067076 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46624f0e6b22475fb447ea07754ef529'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:15.068263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46624f0e6b22475fb447ea07754ef529'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:15.118145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-44117450-7a2c-42e7-b89a-e9073142a7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46624f0e6b22475fb447ea07754ef529'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:15.178123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 829/1675] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:14 2022] DELETE /identity/v3/users/46624f0e6b22475fb447ea07754ef529 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:15.182304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c8751-4bb6-4225-8453-2eba82104bff None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:15.197349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b85f3d-2ed6-4b51-81cf-f44901701c7a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:15.197702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b85f3d-2ed6-4b51-81cf-f44901701c7a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:15.198130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b85f3d-2ed6-4b51-81cf-f44901701c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:15.248284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-12b85f3d-2ed6-4b51-81cf-f44901701c7a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:15.251422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-12b85f3d-2ed6-4b51-81cf-f44901701c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:15.340863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ed39b-dd06-4290-a5c9-37907de66a08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:15.341693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ed39b-dd06-4290-a5c9-37907de66a08 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:15.342581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ed39b-dd06-4290-a5c9-37907de66a08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:15.342581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ed39b-dd06-4290-a5c9-37907de66a08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:15.344546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cd7ed39b-dd06-4290-a5c9-37907de66a08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:15.445470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:15.484131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ed39b-dd06-4290-a5c9-37907de66a08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:15.486233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ed39b-dd06-4290-a5c9-37907de66a08 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:15.491246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:15.494050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 830/1676] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:15.501193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 847/1677] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:15.502672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9af556-57e3-4763-9870-7a7c610710be None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:15.502958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9af556-57e3-4763-9870-7a7c610710be None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:15.503227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9af556-57e3-4763-9870-7a7c610710be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:15.504220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f4509-0d5d-4f98-99a6-150f2dedd4d4 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:15.504675 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f4509-0d5d-4f98-99a6-150f2dedd4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:15.505212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f4509-0d5d-4f98-99a6-150f2dedd4d4 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:15.507343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 848/1678] 10.222.0.78 () {54 vars in 912 bytes} [Tue Aug 30 01:01:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:15.508351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f4509-0d5d-4f98-99a6-150f2dedd4d4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:15.530870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f4509-0d5d-4f98-99a6-150f2dedd4d4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:15.538011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51c58e-8444-41c1-8fdb-609a9688aff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:15.539106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51c58e-8444-41c1-8fdb-609a9688aff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:15.540434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51c58e-8444-41c1-8fdb-609a9688aff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:15.540990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51c58e-8444-41c1-8fdb-609a9688aff1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:15.541979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51c58e-8444-41c1-8fdb-609a9688aff1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:15.542498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51c58e-8444-41c1-8fdb-609a9688aff1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:15.544861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51c58e-8444-41c1-8fdb-609a9688aff1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:15.556395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4d9af556-57e3-4763-9870-7a7c610710be None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:15.560048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4d9af556-57e3-4763-9870-7a7c610710be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:15.568007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 849/1679] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:15.570409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e820c3-1066-419c-8084-38bdefebf663 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:15.571589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e820c3-1066-419c-8084-38bdefebf663 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:15.572154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e820c3-1066-419c-8084-38bdefebf663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:15.588080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 831/1680] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:15.588737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9af556-57e3-4763-9870-7a7c610710be None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:15.614063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9af556-57e3-4763-9870-7a7c610710be None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:15.630602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d428ad-1cd2-4eb5-b824-700eb1c8c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:15.631233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d428ad-1cd2-4eb5-b824-700eb1c8c905 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:15.631517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d428ad-1cd2-4eb5-b824-700eb1c8c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:15.631790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d428ad-1cd2-4eb5-b824-700eb1c8c905 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:15.632941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d428ad-1cd2-4eb5-b824-700eb1c8c905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:15.635375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d428ad-1cd2-4eb5-b824-700eb1c8c905 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:15.681511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f0e820c3-1066-419c-8084-38bdefebf663 None None] MFA Rules not processed for user `1a5d841380c243209f55314b545ff56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:15.683659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f0e820c3-1066-419c-8084-38bdefebf663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:15.715550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 832/1681] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:15 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:15.715550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d428ad-1cd2-4eb5-b824-700eb1c8c905 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:15.726673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-44d428ad-1cd2-4eb5-b824-700eb1c8c905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:15.819243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 850/1682] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:01:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:15.822874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e820c3-1066-419c-8084-38bdefebf663 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:15.853218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e820c3-1066-419c-8084-38bdefebf663 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:15.943338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d428ad-1cd2-4eb5-b824-700eb1c8c905 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:15.959002 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a7edf-b65e-4852-9c71-02ee5df29b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:15.959002 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a7edf-b65e-4852-9c71-02ee5df29b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:15.959002 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a7edf-b65e-4852-9c71-02ee5df29b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:15.959534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a7edf-b65e-4852-9c71-02ee5df29b58 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:15.961408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a7edf-b65e-4852-9c71-02ee5df29b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:15.964089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a7edf-b65e-4852-9c71-02ee5df29b58 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:15.974532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444c39f-ee12-4c11-b4bf-bb0491953ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:15.978525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444c39f-ee12-4c11-b4bf-bb0491953ead service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:15.979636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444c39f-ee12-4c11-b4bf-bb0491953ead service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:15.980314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444c39f-ee12-4c11-b4bf-bb0491953ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:16.004505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444c39f-ee12-4c11-b4bf-bb0491953ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:16.007699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444c39f-ee12-4c11-b4bf-bb0491953ead service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:16.013747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 851/1683] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:15 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:16.014633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a7edf-b65e-4852-9c71-02ee5df29b58 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:16.021063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ce9a7edf-b65e-4852-9c71-02ee5df29b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:16.034004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 833/1684] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:16.035724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444c39f-ee12-4c11-b4bf-bb0491953ead service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:16.038599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a444c39f-ee12-4c11-b4bf-bb0491953ead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:16.065692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444c39f-ee12-4c11-b4bf-bb0491953ead service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:16.075143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa95160-4357-4499-bc2e-6ba616dfd4d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:16.076353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa95160-4357-4499-bc2e-6ba616dfd4d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:16.077637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa95160-4357-4499-bc2e-6ba616dfd4d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:16.078376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa95160-4357-4499-bc2e-6ba616dfd4d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:16.094482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa95160-4357-4499-bc2e-6ba616dfd4d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:16.097596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa95160-4357-4499-bc2e-6ba616dfd4d9 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:16.111229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 834/1685] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:16.112655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa95160-4357-4499-bc2e-6ba616dfd4d9 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:16.128207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa95160-4357-4499-bc2e-6ba616dfd4d9 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:16.134412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdcd32-ce24-4084-bd59-f8916d2ae9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:16.135270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdcd32-ce24-4084-bd59-f8916d2ae9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:16.135655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdcd32-ce24-4084-bd59-f8916d2ae9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:16.136192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdcd32-ce24-4084-bd59-f8916d2ae9f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:16.136740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdcd32-ce24-4084-bd59-f8916d2ae9f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:16.138700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdcd32-ce24-4084-bd59-f8916d2ae9f6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:16.153958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 835/1686] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:16.157825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdcd32-ce24-4084-bd59-f8916d2ae9f6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:16.170919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdcd32-ce24-4084-bd59-f8916d2ae9f6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:16.176828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a7edf-b65e-4852-9c71-02ee5df29b58 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:16.179062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3c27-f71d-4a11-b584-59bb49acb0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:16.179778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3c27-f71d-4a11-b584-59bb49acb0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:16.180154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3c27-f71d-4a11-b584-59bb49acb0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:16.180614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3c27-f71d-4a11-b584-59bb49acb0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:16.181475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e3c27-f71d-4a11-b584-59bb49acb0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:16.182789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80f3c4-e146-4b0b-90dd-b9adbfb0f0af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:16.183286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e3c27-f71d-4a11-b584-59bb49acb0c6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:16.184127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80f3c4-e146-4b0b-90dd-b9adbfb0f0af service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:16.184555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80f3c4-e146-4b0b-90dd-b9adbfb0f0af service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:16.185177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80f3c4-e146-4b0b-90dd-b9adbfb0f0af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:16.187553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3d80f3c4-e146-4b0b-90dd-b9adbfb0f0af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:16.197433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 836/1687] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:01:16.202527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3c27-f71d-4a11-b584-59bb49acb0c6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:16.219603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3c27-f71d-4a11-b584-59bb49acb0c6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:16.222045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:16.242697 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8a1f7-62d6-49f5-b5b8-5dd09249c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:16.243382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8a1f7-62d6-49f5-b5b8-5dd09249c9ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:16.243693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8a1f7-62d6-49f5-b5b8-5dd09249c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:16.243923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8a1f7-62d6-49f5-b5b8-5dd09249c9ec admin admin] PATH_INFO: `/v3/projects/155c299887374425a02501a6fa2d987d/users/bb0c3a8ee63549b3895e178d2e477e75/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:16.270571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8a1f7-62d6-49f5-b5b8-5dd09249c9ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=155c299887374425a02501a6fa2d987d, user_id=bb0c3a8ee63549b3895e178d2e477e75, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:16.275122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8a1f7-62d6-49f5-b5b8-5dd09249c9ec admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:16.301499 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 837/1688] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:16 2022] PUT /identity/v3/projects/155c299887374425a02501a6fa2d987d/users/bb0c3a8ee63549b3895e178d2e477e75/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:16.313227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b4d8e-ae82-42d6-a69c-c89952fad35f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:16.313618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b4d8e-ae82-42d6-a69c-c89952fad35f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:16.313900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b4d8e-ae82-42d6-a69c-c89952fad35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:16.321629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80f3c4-e146-4b0b-90dd-b9adbfb0f0af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:16.324140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80f3c4-e146-4b0b-90dd-b9adbfb0f0af service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:16.352943 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 852/1689] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:16.389836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-526b4d8e-ae82-42d6-a69c-c89952fad35f None None] MFA Rules not processed for user `bb0c3a8ee63549b3895e178d2e477e75`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:16.391399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-526b4d8e-ae82-42d6-a69c-c89952fad35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:16.517024 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:16.532624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 838/1690] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:16.534169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80f3c4-e146-4b0b-90dd-b9adbfb0f0af service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:16.556718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80f3c4-e146-4b0b-90dd-b9adbfb0f0af service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:16.725060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c0dc7-f9f9-4630-aa5a-07efe667156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:16.725967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c0dc7-f9f9-4630-aa5a-07efe667156f admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:16.726248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c0dc7-f9f9-4630-aa5a-07efe667156f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:16.726880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c0dc7-f9f9-4630-aa5a-07efe667156f admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:16.727731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c0dc7-f9f9-4630-aa5a-07efe667156f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:16.730597 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c0dc7-f9f9-4630-aa5a-07efe667156f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:16.789206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 853/1691] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:16 2022] POST /identity/v3/projects => generated 367 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:16.790429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b4d8e-ae82-42d6-a69c-c89952fad35f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:16.808333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b4d8e-ae82-42d6-a69c-c89952fad35f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:17.039010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dafff-41ba-4e3e-be9d-a878a976a7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:17.039564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dafff-41ba-4e3e-be9d-a878a976a7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:17.039956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dafff-41ba-4e3e-be9d-a878a976a7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:17.040328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dafff-41ba-4e3e-be9d-a878a976a7b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:17.041281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dafff-41ba-4e3e-be9d-a878a976a7b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:17.049869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c1c79-5b98-4312-874c-f010f9bcf03e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:17.050865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c1c79-5b98-4312-874c-f010f9bcf03e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:17.051271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dafff-41ba-4e3e-be9d-a878a976a7b9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:17.051718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c1c79-5b98-4312-874c-f010f9bcf03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:17.097197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 839/1692] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:16 2022] POST /identity/v3/users => generated 290 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:17.104444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dafff-41ba-4e3e-be9d-a878a976a7b9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:17.135502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dafff-41ba-4e3e-be9d-a878a976a7b9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:17.159549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeba8fd-8950-4537-841a-502ef13a7ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:17.160973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeba8fd-8950-4537-841a-502ef13a7ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:17.161517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeba8fd-8950-4537-841a-502ef13a7ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:17.162566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeba8fd-8950-4537-841a-502ef13a7ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:17.163295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4d2c1c79-5b98-4312-874c-f010f9bcf03e None None] MFA Rules not processed for user `c6fe0e0033de447da70be9bc0228eba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:17.163678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeba8fd-8950-4537-841a-502ef13a7ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:17.164530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4d2c1c79-5b98-4312-874c-f010f9bcf03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:17.166185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeba8fd-8950-4537-841a-502ef13a7ac4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:17.229331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 840/1693] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:17.234188 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeba8fd-8950-4537-841a-502ef13a7ac4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:17.287323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeba8fd-8950-4537-841a-502ef13a7ac4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:17.300842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e74ba-2b68-4848-a49c-138aa581b13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:17.301520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e74ba-2b68-4848-a49c-138aa581b13f admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:17.301900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e74ba-2b68-4848-a49c-138aa581b13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:17.302435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e74ba-2b68-4848-a49c-138aa581b13f admin admin] PATH_INFO: `/v3/projects/c6ce0b24324c42b8ae46a3fef5fa9978/users/2e6696c756d94aa7b3f03175cadab919/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:17.355010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e74ba-2b68-4848-a49c-138aa581b13f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ce0b24324c42b8ae46a3fef5fa9978, user_id=2e6696c756d94aa7b3f03175cadab919, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:17.359652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e74ba-2b68-4848-a49c-138aa581b13f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:17.391130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 841/1694] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:17 2022] PUT /identity/v3/projects/c6ce0b24324c42b8ae46a3fef5fa9978/users/2e6696c756d94aa7b3f03175cadab919/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:17.397518 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e74ba-2b68-4848-a49c-138aa581b13f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:17.403263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 854/1695] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:17.416961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c1c79-5b98-4312-874c-f010f9bcf03e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:17.419085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4d2c1c79-5b98-4312-874c-f010f9bcf03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:17.576831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e74ba-2b68-4848-a49c-138aa581b13f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:17.590044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364cf68-0fe3-4869-85c5-87287a2ee6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:17.590948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364cf68-0fe3-4869-85c5-87287a2ee6ca admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:17.591403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364cf68-0fe3-4869-85c5-87287a2ee6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:17.591812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364cf68-0fe3-4869-85c5-87287a2ee6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:17.592609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364cf68-0fe3-4869-85c5-87287a2ee6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:17.594660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364cf68-0fe3-4869-85c5-87287a2ee6ca admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:17.600345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:17.621702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 842/1696] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:17.623522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364cf68-0fe3-4869-85c5-87287a2ee6ca admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:17.626809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b364cf68-0fe3-4869-85c5-87287a2ee6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:17.702847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c1c79-5b98-4312-874c-f010f9bcf03e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:17.827045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:17.850084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e987e-4743-49b0-a41c-ca745aa7c3e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:17.851072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e987e-4743-49b0-a41c-ca745aa7c3e5 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:17.853473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e987e-4743-49b0-a41c-ca745aa7c3e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:17.854913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e987e-4743-49b0-a41c-ca745aa7c3e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:17.884992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364cf68-0fe3-4869-85c5-87287a2ee6ca admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:17.954760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba6b10-c5c9-4baf-8449-231d45f8c73d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:17.956726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba6b10-c5c9-4baf-8449-231d45f8c73d service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:17.962729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba6b10-c5c9-4baf-8449-231d45f8c73d service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:17.962989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba6b10-c5c9-4baf-8449-231d45f8c73d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:17.972428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-25ba6b10-c5c9-4baf-8449-231d45f8c73d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:18.026798 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e987e-4743-49b0-a41c-ca745aa7c3e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:18.032204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba6b10-c5c9-4baf-8449-231d45f8c73d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:18.038607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba6b10-c5c9-4baf-8449-231d45f8c73d service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:18.053235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e987e-4743-49b0-a41c-ca745aa7c3e5 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:18.077519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 843/1697] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:18.078270 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba6b10-c5c9-4baf-8449-231d45f8c73d service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:18.091885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 855/1698] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:18.093239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ec749-f53c-4f47-aab6-587d77c1a1a4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:18.093523 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ec749-f53c-4f47-aab6-587d77c1a1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:18.093793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ec749-f53c-4f47-aab6-587d77c1a1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:18.139969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba6b10-c5c9-4baf-8449-231d45f8c73d service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:18.154173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d5ee2-33a2-4d7e-9cf2-9fdca42aa993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:18.154860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d5ee2-33a2-4d7e-9cf2-9fdca42aa993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:18.155207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d5ee2-33a2-4d7e-9cf2-9fdca42aa993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:18.155504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d5ee2-33a2-4d7e-9cf2-9fdca42aa993 admin admin] PATH_INFO: `/v3/projects/c6ce0b24324c42b8ae46a3fef5fa9978/users/2e6696c756d94aa7b3f03175cadab919/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:18.163560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d5ee2-33a2-4d7e-9cf2-9fdca42aa993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ce0b24324c42b8ae46a3fef5fa9978, user_id=2e6696c756d94aa7b3f03175cadab919, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:18.166347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d5ee2-33a2-4d7e-9cf2-9fdca42aa993 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:18.166428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-bd0ec749-f53c-4f47-aab6-587d77c1a1a4 None None] MFA Rules not processed for user `5c95c8ce27fd43c6a6fe10443b6bde13`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:18.169956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bd0ec749-f53c-4f47-aab6-587d77c1a1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:18.212317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 844/1699] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:18 2022] PUT /identity/v3/projects/c6ce0b24324c42b8ae46a3fef5fa9978/users/2e6696c756d94aa7b3f03175cadab919/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:18.217815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d5ee2-33a2-4d7e-9cf2-9fdca42aa993 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:18.233323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-386d5ee2-33a2-4d7e-9cf2-9fdca42aa993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:18.397301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 856/1700] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:18.399834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb267f-dfa2-47f5-960a-fee1ac09f4b7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:18.415387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb267f-dfa2-47f5-960a-fee1ac09f4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:18.416058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb267f-dfa2-47f5-960a-fee1ac09f4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:18.506224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d5ee2-33a2-4d7e-9cf2-9fdca42aa993 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:18.521151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-edcb267f-dfa2-47f5-960a-fee1ac09f4b7 None None] MFA Rules not processed for user `2e6696c756d94aa7b3f03175cadab919`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:18.522538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-edcb267f-dfa2-47f5-960a-fee1ac09f4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:18.524490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:18.525170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:18.525458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:18.529994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] PATH_INFO: `/v3/projects/276fb73b4bb04c41ac33daed2e04337a` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:18.532112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] RBAC: Authorizing `identity:delete_project(project_id=276fb73b4bb04c41ac33daed2e04337a)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:18.542513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:18.543693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:18.692614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 857/1701] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:18.704371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '276fb73b4bb04c41ac33daed2e04337a', 'request_id': 'req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:18.706387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb267f-dfa2-47f5-960a-fee1ac09f4b7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:18.709564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-edcb267f-dfa2-47f5-960a-fee1ac09f4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:18.743305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '276fb73b4bb04c41ac33daed2e04337a', 'request_id': 'req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:18.758172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '276fb73b4bb04c41ac33daed2e04337a', 'request_id': 'req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:18.774264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] The token cache is being invalidate because project 276fb73b4bb04c41ac33daed2e04337a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:18.774671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:18.777910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6c2a0a2d-4691-4db4-ac3c-1810c9fc560c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:18.780243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 845/1702] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:18 2022] DELETE /identity/v3/projects/276fb73b4bb04c41ac33daed2e04337a => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:18.789463 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018619d4-85ae-406b-907d-c9cb8d04b542 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:18.789937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018619d4-85ae-406b-907d-c9cb8d04b542 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:18.790311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018619d4-85ae-406b-907d-c9cb8d04b542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:18.818022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-018619d4-85ae-406b-907d-c9cb8d04b542 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:18.819373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-018619d4-85ae-406b-907d-c9cb8d04b542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:18.891329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb267f-dfa2-47f5-960a-fee1ac09f4b7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:18.925486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:18.956481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 846/1703] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:18.958104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a97454-2abd-4a04-9b93-2583f775dd11 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:18.958104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a97454-2abd-4a04-9b93-2583f775dd11 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:18.958344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a97454-2abd-4a04-9b93-2583f775dd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.015242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd898e-06a7-4b5f-92a4-ee3c1dae15bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:19.019400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd898e-06a7-4b5f-92a4-ee3c1dae15bb service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.019912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd898e-06a7-4b5f-92a4-ee3c1dae15bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.020423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd898e-06a7-4b5f-92a4-ee3c1dae15bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.021434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a6a97454-2abd-4a04-9b93-2583f775dd11 None None] MFA Rules not processed for user `e4cc89946c2e45a0a75eef16cb042758`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:19.022669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a6a97454-2abd-4a04-9b93-2583f775dd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:19.023199 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-88dd898e-06a7-4b5f-92a4-ee3c1dae15bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:19.168802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 847/1704] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:19.170069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec169f9-724d-4fa2-aa4a-c987f8980cc5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.170360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec169f9-724d-4fa2-aa4a-c987f8980cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.170634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec169f9-724d-4fa2-aa4a-c987f8980cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.191987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd898e-06a7-4b5f-92a4-ee3c1dae15bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:19.194087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd898e-06a7-4b5f-92a4-ee3c1dae15bb service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:19.226578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 858/1705] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:19.231892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14847171-52b4-474e-8499-5f3ae95e6f5d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.236968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14847171-52b4-474e-8499-5f3ae95e6f5d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.248664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14847171-52b4-474e-8499-5f3ae95e6f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.266391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-aec169f9-724d-4fa2-aa4a-c987f8980cc5 None None] MFA Rules not processed for user `cc7cf306aadb45d1b65252b53c86b612`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:19.268574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-aec169f9-724d-4fa2-aa4a-c987f8980cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:19.331622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-14847171-52b4-474e-8499-5f3ae95e6f5d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:19.332903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-14847171-52b4-474e-8499-5f3ae95e6f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:19.359698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 859/1706] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:19.381365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625c3a9-79d3-4806-9030-6c1e97c1061b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.382138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625c3a9-79d3-4806-9030-6c1e97c1061b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.382138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625c3a9-79d3-4806-9030-6c1e97c1061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.410319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:19.447223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c625c3a9-79d3-4806-9030-6c1e97c1061b None None] MFA Rules not processed for user `e4cc89946c2e45a0a75eef16cb042758`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:19.448240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c625c3a9-79d3-4806-9030-6c1e97c1061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:19.478957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 860/1707] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:19.483136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625c3a9-79d3-4806-9030-6c1e97c1061b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:19.489660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c625c3a9-79d3-4806-9030-6c1e97c1061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:19.519292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 848/1708] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:19.523053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec169f9-724d-4fa2-aa4a-c987f8980cc5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:19.541623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625c3a9-79d3-4806-9030-6c1e97c1061b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:19.556575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfbb96-629d-47e9-8fcc-b35945853bcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:19.557300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfbb96-629d-47e9-8fcc-b35945853bcb service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.557496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfbb96-629d-47e9-8fcc-b35945853bcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.557766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfbb96-629d-47e9-8fcc-b35945853bcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.557887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec169f9-724d-4fa2-aa4a-c987f8980cc5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:19.562633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9ccfbb96-629d-47e9-8fcc-b35945853bcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:19.563226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131df54-6d3f-4516-b93e-69c9507b2244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:19.564047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131df54-6d3f-4516-b93e-69c9507b2244 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.564396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131df54-6d3f-4516-b93e-69c9507b2244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.564824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131df54-6d3f-4516-b93e-69c9507b2244 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.565681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131df54-6d3f-4516-b93e-69c9507b2244 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:19.566616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131df54-6d3f-4516-b93e-69c9507b2244 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:19.569175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131df54-6d3f-4516-b93e-69c9507b2244 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:19.587223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 849/1709] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:19.589972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbd8cd-1907-4616-b6c0-81188b681ae5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.590409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbd8cd-1907-4616-b6c0-81188b681ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.590749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbd8cd-1907-4616-b6c0-81188b681ae5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.592850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 850/1710] 10.222.0.78 () {54 vars in 912 bytes} [Tue Aug 30 01:01:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:19.595162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbd8cd-1907-4616-b6c0-81188b681ae5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:19.598339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4bdbd8cd-1907-4616-b6c0-81188b681ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:19.715932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:19.751945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfbb96-629d-47e9-8fcc-b35945853bcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:19.752891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbd8cd-1907-4616-b6c0-81188b681ae5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:19.755355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfbb96-629d-47e9-8fcc-b35945853bcb service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:19.774402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281e3a-a96f-420d-9a6b-aaee44500abd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:19.775708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 861/1711] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:19.777520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281e3a-a96f-420d-9a6b-aaee44500abd service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.778353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281e3a-a96f-420d-9a6b-aaee44500abd service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.779191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281e3a-a96f-420d-9a6b-aaee44500abd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.796693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfbb96-629d-47e9-8fcc-b35945853bcb service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:19.823230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2281e3a-a96f-420d-9a6b-aaee44500abd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:19.830664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2281e3a-a96f-420d-9a6b-aaee44500abd service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:19.834388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfbb96-629d-47e9-8fcc-b35945853bcb service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:19.858332 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c41d8c-d6fb-4719-b62b-a59ddbceeec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:19.858957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c41d8c-d6fb-4719-b62b-a59ddbceeec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.859236 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c41d8c-d6fb-4719-b62b-a59ddbceeec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.859504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c41d8c-d6fb-4719-b62b-a59ddbceeec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.860148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c41d8c-d6fb-4719-b62b-a59ddbceeec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:19.863659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c41d8c-d6fb-4719-b62b-a59ddbceeec2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:19.871679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 851/1712] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:19.881871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8916a-b056-4a43-8ae2-1958fb0e2a8f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.881871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8916a-b056-4a43-8ae2-1958fb0e2a8f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.881871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8916a-b056-4a43-8ae2-1958fb0e2a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.945265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 862/1713] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:19 2022] POST /identity/v3/projects => generated 403 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:19.953095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c41d8c-d6fb-4719-b62b-a59ddbceeec2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:19.985104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8fb8916a-b056-4a43-8ae2-1958fb0e2a8f None None] MFA Rules not processed for user `1a5d841380c243209f55314b545ff56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:19.986948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8fb8916a-b056-4a43-8ae2-1958fb0e2a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:20.126221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c41d8c-d6fb-4719-b62b-a59ddbceeec2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:20.142559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7833b-c91d-45ee-be03-3f062b797da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:20.143627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7833b-c91d-45ee-be03-3f062b797da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:20.143997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7833b-c91d-45ee-be03-3f062b797da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:20.144182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7833b-c91d-45ee-be03-3f062b797da4 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:20.144913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7833b-c91d-45ee-be03-3f062b797da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:20.146832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7833b-c91d-45ee-be03-3f062b797da4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:20.232647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 852/1714] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:01:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:20.247348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8916a-b056-4a43-8ae2-1958fb0e2a8f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:20.247548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 863/1715] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:19 2022] POST /identity/v3/users => generated 302 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:20.279553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8916a-b056-4a43-8ae2-1958fb0e2a8f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:20.293835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabda7d-7279-4115-889b-cd0b137dd489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:20.295889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabda7d-7279-4115-889b-cd0b137dd489 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:20.296246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabda7d-7279-4115-889b-cd0b137dd489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:20.300977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabda7d-7279-4115-889b-cd0b137dd489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:20.318929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabda7d-7279-4115-889b-cd0b137dd489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:20.337669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabda7d-7279-4115-889b-cd0b137dd489 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:20.362018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 853/1716] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:20.367873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7833b-c91d-45ee-be03-3f062b797da4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:20.394164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7833b-c91d-45ee-be03-3f062b797da4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:20.425837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01bde6-8341-4632-898f-449eed512356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:20.426475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01bde6-8341-4632-898f-449eed512356 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:20.426757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01bde6-8341-4632-898f-449eed512356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:20.427030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01bde6-8341-4632-898f-449eed512356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:20.427676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01bde6-8341-4632-898f-449eed512356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:20.429322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01bde6-8341-4632-898f-449eed512356 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:20.488968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 864/1717] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:20.491110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabda7d-7279-4115-889b-cd0b137dd489 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:20.508648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01bde6-8341-4632-898f-449eed512356 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:20.510871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-db01bde6-8341-4632-898f-449eed512356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:20.534699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabda7d-7279-4115-889b-cd0b137dd489 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:20.534943 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01bde6-8341-4632-898f-449eed512356 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:20.544529 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4af27-bf7a-4bda-a862-03770929c363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:20.545190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4af27-bf7a-4bda-a862-03770929c363 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:20.545470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4af27-bf7a-4bda-a862-03770929c363 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:20.545743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4af27-bf7a-4bda-a862-03770929c363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:20.571367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fff68-ce8d-442d-a764-337e5a9290c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:20.574279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fff68-ce8d-442d-a764-337e5a9290c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:20.576967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fff68-ce8d-442d-a764-337e5a9290c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:20.577298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fff68-ce8d-442d-a764-337e5a9290c6 admin admin] PATH_INFO: `/v3/projects/5cb3a2f2ea2248c3a456ceeb0d71996e/users/ad02b134dc544143b18e61cdfe98f5ed/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:20.582648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4af27-bf7a-4bda-a862-03770929c363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:20.584493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4af27-bf7a-4bda-a862-03770929c363 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:20.648420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 865/1718] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:01:20.650949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fff68-ce8d-442d-a764-337e5a9290c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb3a2f2ea2248c3a456ceeb0d71996e, user_id=ad02b134dc544143b18e61cdfe98f5ed, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:20.663688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fff68-ce8d-442d-a764-337e5a9290c6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:20.709809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 854/1719] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:20 2022] PUT /identity/v3/projects/5cb3a2f2ea2248c3a456ceeb0d71996e/users/ad02b134dc544143b18e61cdfe98f5ed/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:20.715410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a7c8f-88a8-450b-be6f-5e534b8ca691 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:20.715718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a7c8f-88a8-450b-be6f-5e534b8ca691 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:20.716189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a7c8f-88a8-450b-be6f-5e534b8ca691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:20.790129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-781a7c8f-88a8-450b-be6f-5e534b8ca691 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:20.799161 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-781a7c8f-88a8-450b-be6f-5e534b8ca691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:20.945201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:21.051752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 866/1720] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:21.067459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fff68-ce8d-442d-a764-337e5a9290c6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:21.069733 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-978fff68-ce8d-442d-a764-337e5a9290c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:21.262198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fff68-ce8d-442d-a764-337e5a9290c6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:21.272248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3115af-864e-4d52-a41b-3174b6c646a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:21.273554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3115af-864e-4d52-a41b-3174b6c646a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:21.273991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3115af-864e-4d52-a41b-3174b6c646a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:21.275131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3115af-864e-4d52-a41b-3174b6c646a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:21.310213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3115af-864e-4d52-a41b-3174b6c646a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:21.311732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3115af-864e-4d52-a41b-3174b6c646a3 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:21.329010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 855/1721] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:21.950591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a032f-0dda-4caa-84ad-e530c5409521 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:21.951522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a032f-0dda-4caa-84ad-e530c5409521 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:21.952032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a032f-0dda-4caa-84ad-e530c5409521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:22.120508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b69a032f-0dda-4caa-84ad-e530c5409521 None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:22.128435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b69a032f-0dda-4caa-84ad-e530c5409521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:22.144470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3115af-864e-4d52-a41b-3174b6c646a3 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:22.403615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:22.420818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 867/1722] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:01:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:22.428794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3115af-864e-4d52-a41b-3174b6c646a3 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:22.446021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e841c1-71e0-4390-9fd0-42b6997d661e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:22.447344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e841c1-71e0-4390-9fd0-42b6997d661e service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:22.448666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e841c1-71e0-4390-9fd0-42b6997d661e service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:22.449743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e841c1-71e0-4390-9fd0-42b6997d661e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:22.669030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e841c1-71e0-4390-9fd0-42b6997d661e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:22.670660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e841c1-71e0-4390-9fd0-42b6997d661e service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:22.707309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 856/1723] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:22.725393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a032f-0dda-4caa-84ad-e530c5409521 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:22.727479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b69a032f-0dda-4caa-84ad-e530c5409521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:23.010662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a032f-0dda-4caa-84ad-e530c5409521 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:23.030187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431759a-0f08-435a-a2b6-11cb9afa5b30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:23.030829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431759a-0f08-435a-a2b6-11cb9afa5b30 service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:23.031107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431759a-0f08-435a-a2b6-11cb9afa5b30 service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:23.031372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431759a-0f08-435a-a2b6-11cb9afa5b30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:23.044497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431759a-0f08-435a-a2b6-11cb9afa5b30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:23.046676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431759a-0f08-435a-a2b6-11cb9afa5b30 service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:23.078709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 868/1724] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:01:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:01:23.770940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334fbe-41f0-4697-ac6c-57353e97c456 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:23.771456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334fbe-41f0-4697-ac6c-57353e97c456 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:23.771563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334fbe-41f0-4697-ac6c-57353e97c456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:23.833691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b5334fbe-41f0-4697-ac6c-57353e97c456 None None] MFA Rules not processed for user `c6fe0e0033de447da70be9bc0228eba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:23.837956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b5334fbe-41f0-4697-ac6c-57353e97c456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:24.210459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 857/1725] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:26.344263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e58db-fb66-4c2a-bdcb-b61d69e61f36 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:26.344263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e58db-fb66-4c2a-bdcb-b61d69e61f36 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:26.344263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e58db-fb66-4c2a-bdcb-b61d69e61f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:26.481035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a57e58db-fb66-4c2a-bdcb-b61d69e61f36 None None] MFA Rules not processed for user `ad02b134dc544143b18e61cdfe98f5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:26.487320 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a57e58db-fb66-4c2a-bdcb-b61d69e61f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:26.523877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:26.739579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 869/1726] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:26.743380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334fbe-41f0-4697-ac6c-57353e97c456 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:26.762963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334fbe-41f0-4697-ac6c-57353e97c456 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:26.781801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106010eb-781f-45e1-8499-d75a76a2f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:26.783481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106010eb-781f-45e1-8499-d75a76a2f8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:26.784093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106010eb-781f-45e1-8499-d75a76a2f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:26.784519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106010eb-781f-45e1-8499-d75a76a2f8ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:26.785342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106010eb-781f-45e1-8499-d75a76a2f8ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:26.793050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106010eb-781f-45e1-8499-d75a76a2f8ee admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:26.824385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fb599-c4ce-4c68-a160-f92435bb1a0f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:26.825801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fb599-c4ce-4c68-a160-f92435bb1a0f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:26.826295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fb599-c4ce-4c68-a160-f92435bb1a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:26.900573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 858/1727] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:26 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:26.901570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106010eb-781f-45e1-8499-d75a76a2f8ee admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:26.903751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-588fb599-c4ce-4c68-a160-f92435bb1a0f None None] MFA Rules not processed for user `1a5d841380c243209f55314b545ff56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:26.912366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-588fb599-c4ce-4c68-a160-f92435bb1a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:27.028948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 870/1728] 10.222.0.47 () {58 vars in 1015 bytes} [Tue Aug 30 01:01:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:27.096967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fb599-c4ce-4c68-a160-f92435bb1a0f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:27.110155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fb599-c4ce-4c68-a160-f92435bb1a0f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:27.138536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5fa9c-84f3-4a2a-9dce-95dc4d17bac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:27.149845 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5fa9c-84f3-4a2a-9dce-95dc4d17bac0 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:27.150134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5fa9c-84f3-4a2a-9dce-95dc4d17bac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:27.150417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5fa9c-84f3-4a2a-9dce-95dc4d17bac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:27.179799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5fa9c-84f3-4a2a-9dce-95dc4d17bac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:27.181329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5fa9c-84f3-4a2a-9dce-95dc4d17bac0 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:27.213385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 871/1729] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:27.216578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106010eb-781f-45e1-8499-d75a76a2f8ee admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:27.235272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afc182-9524-4b78-9c1b-5f9eca1f9d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:27.266533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afc182-9524-4b78-9c1b-5f9eca1f9d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:27.269895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afc182-9524-4b78-9c1b-5f9eca1f9d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:27.271706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afc182-9524-4b78-9c1b-5f9eca1f9d8b admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:27.276087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6afc182-9524-4b78-9c1b-5f9eca1f9d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:27.286369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6afc182-9524-4b78-9c1b-5f9eca1f9d8b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:27.368294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5fa9c-84f3-4a2a-9dce-95dc4d17bac0 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:27.383299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 859/1730] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:26 2022] POST /identity/v3/users => generated 309 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:27.402181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5fa9c-84f3-4a2a-9dce-95dc4d17bac0 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:27.422343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a037654-fa00-4e77-ab7c-b9441e3b233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:27.423793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a037654-fa00-4e77-ab7c-b9441e3b233c admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:27.424227 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a037654-fa00-4e77-ab7c-b9441e3b233c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:27.425153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a037654-fa00-4e77-ab7c-b9441e3b233c admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:27.425966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a037654-fa00-4e77-ab7c-b9441e3b233c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:27.429939 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a037654-fa00-4e77-ab7c-b9441e3b233c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:27.508985 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 872/1731] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:27.510294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afc182-9524-4b78-9c1b-5f9eca1f9d8b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:27.529435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afc182-9524-4b78-9c1b-5f9eca1f9d8b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:27.544862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eff7b0-1fc8-4663-ad7d-a2eb3790c2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:27.555896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff7b0-1fc8-4663-ad7d-a2eb3790c2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:27.556247 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff7b0-1fc8-4663-ad7d-a2eb3790c2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:27.556641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff7b0-1fc8-4663-ad7d-a2eb3790c2ae admin admin] PATH_INFO: `/v3/projects/56f1159c3d7a48b08d6ee1e2cf2dece6/users/16e4bfc24f82419abdf0291d04c7f03a/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:27.612932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a037654-fa00-4e77-ab7c-b9441e3b233c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:27.615477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1a037654-fa00-4e77-ab7c-b9441e3b233c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:27.636221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:27.639120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a037654-fa00-4e77-ab7c-b9441e3b233c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:27.650086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:27.652977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:27.653519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:27.653948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] PATH_INFO: `/v3/users/bb0c3a8ee63549b3895e178d2e477e75` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:27.658046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0c3a8ee63549b3895e178d2e477e75)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:27.661217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:27.669357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eff7b0-1fc8-4663-ad7d-a2eb3790c2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f1159c3d7a48b08d6ee1e2cf2dece6, user_id=16e4bfc24f82419abdf0291d04c7f03a, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:27.682286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eff7b0-1fc8-4663-ad7d-a2eb3790c2ae admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:27.729700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 860/1732] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:27 2022] PUT /identity/v3/projects/56f1159c3d7a48b08d6ee1e2cf2dece6/users/16e4bfc24f82419abdf0291d04c7f03a/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:27.733571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eff7b0-1fc8-4663-ad7d-a2eb3790c2ae admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:27.759117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0c3a8ee63549b3895e178d2e477e75'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:27.774996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0c3a8ee63549b3895e178d2e477e75'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:27.786392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0c3a8ee63549b3895e178d2e477e75'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:27.790137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0c3a8ee63549b3895e178d2e477e75'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:27.823418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1522e944-a609-4d48-9fa3-d20fc1a5f7e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0c3a8ee63549b3895e178d2e477e75'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:27.838521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 873/1733] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:27 2022] DELETE /identity/v3/users/bb0c3a8ee63549b3895e178d2e477e75 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:27.842826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7825d-b48b-410d-a718-b08f1812591b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:27.843414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7825d-b48b-410d-a718-b08f1812591b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:27.843979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7825d-b48b-410d-a718-b08f1812591b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:27.891980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-18d7825d-b48b-410d-a718-b08f1812591b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:27.897743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-18d7825d-b48b-410d-a718-b08f1812591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:27.912269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:27.999735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eff7b0-1fc8-4663-ad7d-a2eb3790c2ae admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:28.130052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 874/1734] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:28.209844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7825d-b48b-410d-a718-b08f1812591b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:28.210805 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebf8a-0bc3-4edf-8233-8a64e128dc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:28.212342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebf8a-0bc3-4edf-8233-8a64e128dc7b admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:28.215706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-18d7825d-b48b-410d-a718-b08f1812591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:28.224579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebf8a-0bc3-4edf-8233-8a64e128dc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:28.226464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebf8a-0bc3-4edf-8233-8a64e128dc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:28.227363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddebf8a-0bc3-4edf-8233-8a64e128dc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:28.229574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddebf8a-0bc3-4edf-8233-8a64e128dc7b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:28.259114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 861/1735] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:28.268846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebf8a-0bc3-4edf-8233-8a64e128dc7b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:28.275836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-eddebf8a-0bc3-4edf-8233-8a64e128dc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:28.324898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebf8a-0bc3-4edf-8233-8a64e128dc7b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:28.343554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a5b20-696c-49eb-af73-886c393fcc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:28.344846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a5b20-696c-49eb-af73-886c393fcc3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:28.347174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a5b20-696c-49eb-af73-886c393fcc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:28.347594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a5b20-696c-49eb-af73-886c393fcc3d admin admin] PATH_INFO: `/v3/projects/56f1159c3d7a48b08d6ee1e2cf2dece6/users/16e4bfc24f82419abdf0291d04c7f03a/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:28.354923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a5b20-696c-49eb-af73-886c393fcc3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f1159c3d7a48b08d6ee1e2cf2dece6, user_id=16e4bfc24f82419abdf0291d04c7f03a, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:28.374118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a5b20-696c-49eb-af73-886c393fcc3d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:28.413017 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 862/1736] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:28 2022] PUT /identity/v3/projects/56f1159c3d7a48b08d6ee1e2cf2dece6/users/16e4bfc24f82419abdf0291d04c7f03a/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:28.454975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7825d-b48b-410d-a718-b08f1812591b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:28.642471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7e627-c3b0-46b2-aab9-d5ef658330a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:28.643106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7e627-c3b0-46b2-aab9-d5ef658330a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:28.643384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7e627-c3b0-46b2-aab9-d5ef658330a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:28.643668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7e627-c3b0-46b2-aab9-d5ef658330a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:28.862002 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7e627-c3b0-46b2-aab9-d5ef658330a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:28.863540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7e627-c3b0-46b2-aab9-d5ef658330a1 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:28.906348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 875/1737] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:01:31.943382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4ed3e-f221-4e67-930b-eda2ebe731bd None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:31.944923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4ed3e-f221-4e67-930b-eda2ebe731bd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:31.945467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4ed3e-f221-4e67-930b-eda2ebe731bd None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:31.947675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 863/1738] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:01:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:32.002810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374c62f-5be1-4977-9a83-e5f7421fbf3a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.003941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374c62f-5be1-4977-9a83-e5f7421fbf3a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.004430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374c62f-5be1-4977-9a83-e5f7421fbf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.074312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b374c62f-5be1-4977-9a83-e5f7421fbf3a None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:32.076036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b374c62f-5be1-4977-9a83-e5f7421fbf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:32.114757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:32.340700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 876/1739] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:01:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:32.962044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4ed3e-f221-4e67-930b-eda2ebe731bd None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.964111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-55f4ed3e-f221-4e67-930b-eda2ebe731bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:32.994592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4ed3e-f221-4e67-930b-eda2ebe731bd None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:33.007609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331c616-ef68-4443-848e-877da45dcaaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:33.009327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331c616-ef68-4443-848e-877da45dcaaa service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:33.010064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331c616-ef68-4443-848e-877da45dcaaa service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:33.010534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331c616-ef68-4443-848e-877da45dcaaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:33.044752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331c616-ef68-4443-848e-877da45dcaaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:33.046476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331c616-ef68-4443-848e-877da45dcaaa service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:33.064820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 864/1740] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:33.561330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374c62f-5be1-4977-9a83-e5f7421fbf3a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:33.563935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b374c62f-5be1-4977-9a83-e5f7421fbf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:33.601567 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374c62f-5be1-4977-9a83-e5f7421fbf3a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:33.615522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:33.616445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:33.616878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:33.617329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] PATH_INFO: `/v3/projects/155c299887374425a02501a6fa2d987d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:33.619303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=155c299887374425a02501a6fa2d987d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:33.630141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:33.719846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155c299887374425a02501a6fa2d987d', 'request_id': 'req-83f1e42a-261b-4c52-aaee-c50fb6c346d3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:33.757817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155c299887374425a02501a6fa2d987d', 'request_id': 'req-83f1e42a-261b-4c52-aaee-c50fb6c346d3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:33.782221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155c299887374425a02501a6fa2d987d', 'request_id': 'req-83f1e42a-261b-4c52-aaee-c50fb6c346d3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:33.831418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] The token cache is being invalidate because project 155c299887374425a02501a6fa2d987d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:33.834381 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:33.836113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-83f1e42a-261b-4c52-aaee-c50fb6c346d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:33.846782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 877/1741] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:33 2022] DELETE /identity/v3/projects/155c299887374425a02501a6fa2d987d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:33.850957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331c616-ef68-4443-848e-877da45dcaaa service nova] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:33.899613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331c616-ef68-4443-848e-877da45dcaaa service nova] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:34.145777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:34.148027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:34.148606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:34.149328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] PATH_INFO: `/v3/users/2e6696c756d94aa7b3f03175cadab919` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:34.151132 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6696c756d94aa7b3f03175cadab919)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:34.154621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:34.165843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:34.306564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6696c756d94aa7b3f03175cadab919'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:34.327176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6696c756d94aa7b3f03175cadab919'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:34.327996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6696c756d94aa7b3f03175cadab919'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:34.329486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6696c756d94aa7b3f03175cadab919'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:34.382141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6537bed-0300-47ae-9eee-df6ec77ee068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6696c756d94aa7b3f03175cadab919'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:34.430098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 865/1742] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:33 2022] DELETE /identity/v3/users/2e6696c756d94aa7b3f03175cadab919 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:34.730703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244930-24b2-492f-ad64-2aafbd689672 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:34.731376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244930-24b2-492f-ad64-2aafbd689672 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:34.731376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244930-24b2-492f-ad64-2aafbd689672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:34.777789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2a244930-24b2-492f-ad64-2aafbd689672 None None] MFA Rules not processed for user `cc7cf306aadb45d1b65252b53c86b612`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:34.778953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce76d6a-5885-4fb6-b1a2-d5ae744fde96 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:34.779194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2a244930-24b2-492f-ad64-2aafbd689672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:34.779322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce76d6a-5885-4fb6-b1a2-d5ae744fde96 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:34.779521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce76d6a-5885-4fb6-b1a2-d5ae744fde96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:34.839945 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4ce76d6a-5885-4fb6-b1a2-d5ae744fde96 None None] MFA Rules not processed for user `16e4bfc24f82419abdf0291d04c7f03a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:34.842507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4ce76d6a-5885-4fb6-b1a2-d5ae744fde96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:34.890200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 878/1743] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:34.984007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 866/1744] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:35.770442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244930-24b2-492f-ad64-2aafbd689672 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:35.771897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2a244930-24b2-492f-ad64-2aafbd689672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:35.866033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:35.917779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244930-24b2-492f-ad64-2aafbd689672 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:35.924247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:35.925056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:35.925538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:35.925978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] PATH_INFO: `/v3/projects/c6ce0b24324c42b8ae46a3fef5fa9978` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:35.928483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ce0b24324c42b8ae46a3fef5fa9978)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:35.930860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:36.045319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ce0b24324c42b8ae46a3fef5fa9978', 'request_id': 'req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:36.059915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ce0b24324c42b8ae46a3fef5fa9978', 'request_id': 'req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:36.078366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ce0b24324c42b8ae46a3fef5fa9978', 'request_id': 'req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:36.110120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] The token cache is being invalidate because project c6ce0b24324c42b8ae46a3fef5fa9978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:36.110357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:36.112427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-94e971fd-c0b4-4f3b-9a27-483663ecf8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:36.121214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 879/1745] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:35 2022] DELETE /identity/v3/projects/c6ce0b24324c42b8ae46a3fef5fa9978 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:36.126963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3c4ac-90a0-4874-b4c7-06c789c7b014 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.127508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3c4ac-90a0-4874-b4c7-06c789c7b014 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.127508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3c4ac-90a0-4874-b4c7-06c789c7b014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.182737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e3f3c4ac-90a0-4874-b4c7-06c789c7b014 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:36.185550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e3f3c4ac-90a0-4874-b4c7-06c789c7b014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:36.337253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 867/1746] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:36.340472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e1681-1c85-47f6-8d0e-94aa66404e07 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.340758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e1681-1c85-47f6-8d0e-94aa66404e07 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.341031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e1681-1c85-47f6-8d0e-94aa66404e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.364764 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-163e1681-1c85-47f6-8d0e-94aa66404e07 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:36.366032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-163e1681-1c85-47f6-8d0e-94aa66404e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:36.382752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3c4ac-90a0-4874-b4c7-06c789c7b014 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:36.383223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 880/1747] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:36.393742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3c4ac-90a0-4874-b4c7-06c789c7b014 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:36.400199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd7d6a-8455-4da2-8cc2-fce4947c24b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:36.401117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd7d6a-8455-4da2-8cc2-fce4947c24b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.401342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd7d6a-8455-4da2-8cc2-fce4947c24b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.401872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd7d6a-8455-4da2-8cc2-fce4947c24b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.417621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd7d6a-8455-4da2-8cc2-fce4947c24b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:36.418388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd7d6a-8455-4da2-8cc2-fce4947c24b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:36.419944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd7d6a-8455-4da2-8cc2-fce4947c24b4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:36.428856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:36.437485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 868/1748] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:36.441796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e1681-1c85-47f6-8d0e-94aa66404e07 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:36.455076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e1681-1c85-47f6-8d0e-94aa66404e07 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:36.459905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd3468-da8b-4b47-9e1b-f8ce9836ed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:36.460613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd3468-da8b-4b47-9e1b-f8ce9836ed2c admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.460951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd3468-da8b-4b47-9e1b-f8ce9836ed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.461330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd3468-da8b-4b47-9e1b-f8ce9836ed2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.462110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd3468-da8b-4b47-9e1b-f8ce9836ed2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:36.464562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd3468-da8b-4b47-9e1b-f8ce9836ed2c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:36.513422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 881/1749] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:36 2022] POST /identity/v3/projects => generated 387 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:36.513635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd7d6a-8455-4da2-8cc2-fce4947c24b4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:36.533701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd7d6a-8455-4da2-8cc2-fce4947c24b4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:36.602556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0af2-730c-4c33-b4b3-ecd703759d56 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.602556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0af2-730c-4c33-b4b3-ecd703759d56 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.602759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0af2-730c-4c33-b4b3-ecd703759d56 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.606396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 882/1750] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:01:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:36.638848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a54b39-4bd3-4351-94f0-88be26937328 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.639479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a54b39-4bd3-4351-94f0-88be26937328 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.639955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a54b39-4bd3-4351-94f0-88be26937328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.660611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ac862-dcab-4483-8f07-80f16ebcd16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:36.661052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac862-dcab-4483-8f07-80f16ebcd16c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.662388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac862-dcab-4483-8f07-80f16ebcd16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.662388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac862-dcab-4483-8f07-80f16ebcd16c admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.663058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ac862-dcab-4483-8f07-80f16ebcd16c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:36.665418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ac862-dcab-4483-8f07-80f16ebcd16c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:36.701164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 869/1751] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:36 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:01:36.705232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ac862-dcab-4483-8f07-80f16ebcd16c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:36.705701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-21a54b39-4bd3-4351-94f0-88be26937328 None None] MFA Rules not processed for user `8f860912e1f0406cadd70da440b2ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:36.707269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-21a54b39-4bd3-4351-94f0-88be26937328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:36.721629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ac862-dcab-4483-8f07-80f16ebcd16c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:36.727123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76016628-40cb-4ee9-81ad-a3e6048b81fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:36.727831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76016628-40cb-4ee9-81ad-a3e6048b81fa admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.728145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76016628-40cb-4ee9-81ad-a3e6048b81fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.728366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76016628-40cb-4ee9-81ad-a3e6048b81fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.729044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76016628-40cb-4ee9-81ad-a3e6048b81fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:36.731025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76016628-40cb-4ee9-81ad-a3e6048b81fa admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:36.747445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 870/1752] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:36.748969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76016628-40cb-4ee9-81ad-a3e6048b81fa admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:36.764078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76016628-40cb-4ee9-81ad-a3e6048b81fa admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:36.771524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c393a1-a227-41cf-8791-f19922abf84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:36.772269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c393a1-a227-41cf-8791-f19922abf84b admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.773600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c393a1-a227-41cf-8791-f19922abf84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.773600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c393a1-a227-41cf-8791-f19922abf84b admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.774510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c393a1-a227-41cf-8791-f19922abf84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:36.777000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c393a1-a227-41cf-8791-f19922abf84b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:36.794574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 871/1753] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:36.795911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c393a1-a227-41cf-8791-f19922abf84b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:36.809634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c393a1-a227-41cf-8791-f19922abf84b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:36.852462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da8a2e-1504-401c-959e-27c455947d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:36.854654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da8a2e-1504-401c-959e-27c455947d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.856485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da8a2e-1504-401c-959e-27c455947d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.856853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da8a2e-1504-401c-959e-27c455947d78 admin admin] PATH_INFO: `/v3/projects/965e0d552cd9492595bba638711d2609/users/3fab7f7674b54bb3bcbfd792c428dfd0/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.863200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 883/1754] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:01:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:36.908603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da8a2e-1504-401c-959e-27c455947d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=965e0d552cd9492595bba638711d2609, user_id=3fab7f7674b54bb3bcbfd792c428dfd0, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:36.912169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da8a2e-1504-401c-959e-27c455947d78 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:36.919337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac9140-a9db-4c66-9b97-3e65bb69bcbd None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.925543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac9140-a9db-4c66-9b97-3e65bb69bcbd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.925870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac9140-a9db-4c66-9b97-3e65bb69bcbd None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.928165 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 884/1755] 10.222.0.78 () {54 vars in 961 bytes} [Tue Aug 30 01:01:36 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:36.934456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e35e58-a4ca-48ed-a62a-25c66b3dbfa2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.935153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e35e58-a4ca-48ed-a62a-25c66b3dbfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.935654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e35e58-a4ca-48ed-a62a-25c66b3dbfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.951726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 872/1756] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:36 2022] PUT /identity/v3/projects/965e0d552cd9492595bba638711d2609/users/3fab7f7674b54bb3bcbfd792c428dfd0/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:36.967268 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66617c-9da0-41b8-879f-388caea6b1d3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.967583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66617c-9da0-41b8-879f-388caea6b1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.967958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66617c-9da0-41b8-879f-388caea6b1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.983565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-00e35e58-a4ca-48ed-a62a-25c66b3dbfa2 None None] MFA Rules not processed for user `8f860912e1f0406cadd70da440b2ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:36.986971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-00e35e58-a4ca-48ed-a62a-25c66b3dbfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:37.065056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:37.079976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ee66617c-9da0-41b8-879f-388caea6b1d3 None None] MFA Rules not processed for user `3fab7f7674b54bb3bcbfd792c428dfd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:37.082753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ee66617c-9da0-41b8-879f-388caea6b1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:37.127064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 885/1757] 10.222.0.78 () {58 vars in 1058 bytes} [Tue Aug 30 01:01:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:37.134825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e35e58-a4ca-48ed-a62a-25c66b3dbfa2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:37.155593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e35e58-a4ca-48ed-a62a-25c66b3dbfa2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:37.163387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f938e41-495c-46c1-8996-2f92c6901698 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:37.164197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f938e41-495c-46c1-8996-2f92c6901698 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:37.164580 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f938e41-495c-46c1-8996-2f92c6901698 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:37.165188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f938e41-495c-46c1-8996-2f92c6901698 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:37.178001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f938e41-495c-46c1-8996-2f92c6901698 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:37.180874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f938e41-495c-46c1-8996-2f92c6901698 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:37.202643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 886/1758] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:37.267281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 873/1759] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:37.268683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f938e41-495c-46c1-8996-2f92c6901698 service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:37.408630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f938e41-495c-46c1-8996-2f92c6901698 service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:37.417296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa3156-e560-49f0-81ff-421ceb81a659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:37.418994 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa3156-e560-49f0-81ff-421ceb81a659 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:37.419435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa3156-e560-49f0-81ff-421ceb81a659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:37.419908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa3156-e560-49f0-81ff-421ceb81a659 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:37.420698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa3156-e560-49f0-81ff-421ceb81a659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:37.425392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa3156-e560-49f0-81ff-421ceb81a659 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:37.494283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 887/1760] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:37 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:37.496027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66617c-9da0-41b8-879f-388caea6b1d3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:37.538180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66617c-9da0-41b8-879f-388caea6b1d3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:37.624412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:37.697984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeab95f-27ca-4a98-aa17-d18b31c06236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:37.698695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeab95f-27ca-4a98-aa17-d18b31c06236 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:37.699140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeab95f-27ca-4a98-aa17-d18b31c06236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:37.699394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeab95f-27ca-4a98-aa17-d18b31c06236 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:37.700284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeab95f-27ca-4a98-aa17-d18b31c06236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:37.704511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeab95f-27ca-4a98-aa17-d18b31c06236 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:37.734620 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 874/1761] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:37 2022] POST /identity/v3/users => generated 300 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:37.742691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa3156-e560-49f0-81ff-421ceb81a659 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:37.757842 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa3156-e560-49f0-81ff-421ceb81a659 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:37.766155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878e233-2440-4c8e-848a-51079db3c851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:37.767352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878e233-2440-4c8e-848a-51079db3c851 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:37.767727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878e233-2440-4c8e-848a-51079db3c851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:37.768168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878e233-2440-4c8e-848a-51079db3c851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:37.769455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878e233-2440-4c8e-848a-51079db3c851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:37.771286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878e233-2440-4c8e-848a-51079db3c851 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:37.789483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 888/1762] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:37.794667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeab95f-27ca-4a98-aa17-d18b31c06236 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:37.809616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeab95f-27ca-4a98-aa17-d18b31c06236 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:37.815345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa4189-bba8-43b0-86f5-3d0ff1d2fc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:37.816013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa4189-bba8-43b0-86f5-3d0ff1d2fc2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:37.816293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa4189-bba8-43b0-86f5-3d0ff1d2fc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:37.816566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa4189-bba8-43b0-86f5-3d0ff1d2fc2a admin admin] PATH_INFO: `/v3/projects/116ffc7fd32f41c3bdf8218e3a8de537/users/6c876432dba9499da480c151af4fe6ca/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:37.839315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa4189-bba8-43b0-86f5-3d0ff1d2fc2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=116ffc7fd32f41c3bdf8218e3a8de537, user_id=6c876432dba9499da480c151af4fe6ca, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:37.851732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa4189-bba8-43b0-86f5-3d0ff1d2fc2a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:37.884474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 875/1763] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:37 2022] PUT /identity/v3/projects/116ffc7fd32f41c3bdf8218e3a8de537/users/6c876432dba9499da480c151af4fe6ca/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:37.885725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878e233-2440-4c8e-848a-51079db3c851 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:37.902486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878e233-2440-4c8e-848a-51079db3c851 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:38.008612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37168b-14d6-428c-b165-a1dc7ed2d359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:38.009313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37168b-14d6-428c-b165-a1dc7ed2d359 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:38.009592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37168b-14d6-428c-b165-a1dc7ed2d359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:38.010038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37168b-14d6-428c-b165-a1dc7ed2d359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:38.010749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37168b-14d6-428c-b165-a1dc7ed2d359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:38.012573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37168b-14d6-428c-b165-a1dc7ed2d359 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:38.030184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 889/1764] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:38.035710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa4189-bba8-43b0-86f5-3d0ff1d2fc2a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:38.081951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa4189-bba8-43b0-86f5-3d0ff1d2fc2a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:38.096447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc90eae-bc24-466c-a4f6-f7bc5a44f38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:38.097262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc90eae-bc24-466c-a4f6-f7bc5a44f38b admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:38.097547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc90eae-bc24-466c-a4f6-f7bc5a44f38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:38.097839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc90eae-bc24-466c-a4f6-f7bc5a44f38b admin admin] PATH_INFO: `/v3/projects/116ffc7fd32f41c3bdf8218e3a8de537/users/6c876432dba9499da480c151af4fe6ca/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:38.108468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc90eae-bc24-466c-a4f6-f7bc5a44f38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=116ffc7fd32f41c3bdf8218e3a8de537, user_id=6c876432dba9499da480c151af4fe6ca, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:38.113253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc90eae-bc24-466c-a4f6-f7bc5a44f38b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:38.146105 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 876/1765] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:38 2022] PUT /identity/v3/projects/116ffc7fd32f41c3bdf8218e3a8de537/users/6c876432dba9499da480c151af4fe6ca/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:38.159221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e247a-5c62-4f66-9b9c-eb431e74652f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:38.159673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e247a-5c62-4f66-9b9c-eb431e74652f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:38.160084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e247a-5c62-4f66-9b9c-eb431e74652f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:38.161898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc90eae-bc24-466c-a4f6-f7bc5a44f38b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:38.163657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3cc90eae-bc24-466c-a4f6-f7bc5a44f38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:38.244925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4a4e247a-5c62-4f66-9b9c-eb431e74652f None None] MFA Rules not processed for user `6c876432dba9499da480c151af4fe6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:38.245378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4a4e247a-5c62-4f66-9b9c-eb431e74652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:38.273158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:38.360465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc90eae-bc24-466c-a4f6-f7bc5a44f38b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:38.365429 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 890/1766] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:38.365684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:38.366327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:38.366638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:38.367065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] PATH_INFO: `/v3/users/ad02b134dc544143b18e61cdfe98f5ed` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:38.368387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad02b134dc544143b18e61cdfe98f5ed)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:38.370437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:38.382758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e247a-5c62-4f66-9b9c-eb431e74652f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:38.384209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4a4e247a-5c62-4f66-9b9c-eb431e74652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:38.457878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e247a-5c62-4f66-9b9c-eb431e74652f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:38.461865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae61c0-94cb-4d22-bf5d-8cce48490de5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:38.462595 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae61c0-94cb-4d22-bf5d-8cce48490de5 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:38.462975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae61c0-94cb-4d22-bf5d-8cce48490de5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:38.463473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae61c0-94cb-4d22-bf5d-8cce48490de5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:38.474230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad02b134dc544143b18e61cdfe98f5ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.474859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae61c0-94cb-4d22-bf5d-8cce48490de5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:38.476520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae61c0-94cb-4d22-bf5d-8cce48490de5 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:38.490048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad02b134dc544143b18e61cdfe98f5ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.490801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad02b134dc544143b18e61cdfe98f5ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.491491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad02b134dc544143b18e61cdfe98f5ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.501599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 891/1767] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:38.541171 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40e49bd7-7397-4aa6-b758-9f79440e84b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad02b134dc544143b18e61cdfe98f5ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.584749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae61c0-94cb-4d22-bf5d-8cce48490de5 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:38.601509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae61c0-94cb-4d22-bf5d-8cce48490de5 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:38.601835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 877/1768] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:38 2022] DELETE /identity/v3/users/ad02b134dc544143b18e61cdfe98f5ed => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:38.701479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:38.702623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:38.702623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:38.702804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] PATH_INFO: `/v3/users/3fab7f7674b54bb3bcbfd792c428dfd0` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:38.703874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fab7f7674b54bb3bcbfd792c428dfd0)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:38.706142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:38.787022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fab7f7674b54bb3bcbfd792c428dfd0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.801633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fab7f7674b54bb3bcbfd792c428dfd0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.804230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:38.812778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fab7f7674b54bb3bcbfd792c428dfd0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.815130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fab7f7674b54bb3bcbfd792c428dfd0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.856676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fab7f7674b54bb3bcbfd792c428dfd0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:38.870345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 892/1769] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:38 2022] DELETE /identity/v3/users/3fab7f7674b54bb3bcbfd792c428dfd0 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:38.874996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d651f4-c208-4c53-9045-c88508e4fc05 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:38.875672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d651f4-c208-4c53-9045-c88508e4fc05 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:38.875838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d651f4-c208-4c53-9045-c88508e4fc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:38.884976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:38.900519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a3d651f4-c208-4c53-9045-c88508e4fc05 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:38.902314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a3d651f4-c208-4c53-9045-c88508e4fc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:39.005279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 878/1770] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:39.017365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:39.017365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:39.105485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d651f4-c208-4c53-9045-c88508e4fc05 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:39.106937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a3d651f4-c208-4c53-9045-c88508e4fc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:39.128173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510bd22-b4f7-496e-87be-9edadf8c6d01 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:39.129230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d651f4-c208-4c53-9045-c88508e4fc05 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:39.132317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9206d7f-570f-44e6-b92d-77a48bad42f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:39.132891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9206d7f-570f-44e6-b92d-77a48bad42f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:39.133253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9206d7f-570f-44e6-b92d-77a48bad42f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:39.133457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9206d7f-570f-44e6-b92d-77a48bad42f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:39.134669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:39.135361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:39.135690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:39.135987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] PATH_INFO: `/v3/projects/5cb3a2f2ea2248c3a456ceeb0d71996e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:39.138838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb3a2f2ea2248c3a456ceeb0d71996e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:39.139417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:39.147318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9206d7f-570f-44e6-b92d-77a48bad42f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:39.148863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9206d7f-570f-44e6-b92d-77a48bad42f3 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:39.163691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 893/1771] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:39.205421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb3a2f2ea2248c3a456ceeb0d71996e', 'request_id': 'req-180484d7-845d-45fb-805f-be294f523637'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.212872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb3a2f2ea2248c3a456ceeb0d71996e', 'request_id': 'req-180484d7-845d-45fb-805f-be294f523637'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.224705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb3a2f2ea2248c3a456ceeb0d71996e', 'request_id': 'req-180484d7-845d-45fb-805f-be294f523637'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.243547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] The token cache is being invalidate because project 5cb3a2f2ea2248c3a456ceeb0d71996e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:39.244139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.245176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.247575 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 879/1772] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:39 2022] DELETE /identity/v3/projects/5cb3a2f2ea2248c3a456ceeb0d71996e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:39.252244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9206d7f-570f-44e6-b92d-77a48bad42f3 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:39.266083 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9206d7f-570f-44e6-b92d-77a48bad42f3 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:39.361479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:39.362222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:39.362524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:39.362929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] PATH_INFO: `/v3/users/16e4bfc24f82419abdf0291d04c7f03a` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:39.366082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e4bfc24f82419abdf0291d04c7f03a)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:39.368117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:39.439450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e4bfc24f82419abdf0291d04c7f03a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.449667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e4bfc24f82419abdf0291d04c7f03a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.462572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e4bfc24f82419abdf0291d04c7f03a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.466530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e4bfc24f82419abdf0291d04c7f03a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.479294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:39.480632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e4bfc24f82419abdf0291d04c7f03a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.490469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 894/1773] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:39 2022] DELETE /identity/v3/users/16e4bfc24f82419abdf0291d04c7f03a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:39.580243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:39.581826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:39.595569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180484d7-845d-45fb-805f-be294f523637 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:39.673644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:39.674281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:39.674558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:39.674830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] PATH_INFO: `/v3/projects/965e0d552cd9492595bba638711d2609` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:39.675976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=965e0d552cd9492595bba638711d2609)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:39.677933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:39.718166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:39.718749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:39.721582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:39.738777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965e0d552cd9492595bba638711d2609', 'request_id': 'req-2688efa3-045c-4f23-a9c0-7dc8631beb9d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.740864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754726d-9ac5-46de-a2b3-800a15983690 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:39.745832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965e0d552cd9492595bba638711d2609', 'request_id': 'req-2688efa3-045c-4f23-a9c0-7dc8631beb9d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.759311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965e0d552cd9492595bba638711d2609', 'request_id': 'req-2688efa3-045c-4f23-a9c0-7dc8631beb9d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.774508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] The token cache is being invalidate because project 965e0d552cd9492595bba638711d2609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:39.774804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.775518 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.780205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 880/1774] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:39 2022] DELETE /identity/v3/projects/965e0d552cd9492595bba638711d2609 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:39.783212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:39.784909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:39.832158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:39.841417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:39.841785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:39.842128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] PATH_INFO: `/v3/projects/56f1159c3d7a48b08d6ee1e2cf2dece6` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:39.843417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f1159c3d7a48b08d6ee1e2cf2dece6)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:39.846289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:39.885099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688efa3-045c-4f23-a9c0-7dc8631beb9d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:39.894811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f1159c3d7a48b08d6ee1e2cf2dece6', 'request_id': 'req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.901267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f1159c3d7a48b08d6ee1e2cf2dece6', 'request_id': 'req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.910052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f1159c3d7a48b08d6ee1e2cf2dece6', 'request_id': 'req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.922033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] The token cache is being invalidate because project 56f1159c3d7a48b08d6ee1e2cf2dece6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:39.923023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.923999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ccb7ad0d-cc6a-4953-b6ac-3ed1c8723940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.926591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 895/1775] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:39 2022] DELETE /identity/v3/projects/56f1159c3d7a48b08d6ee1e2cf2dece6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:39.934194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e71ad-442c-4826-b77f-48c5ec8b3f98 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:39.934194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e71ad-442c-4826-b77f-48c5ec8b3f98 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:39.934427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e71ad-442c-4826-b77f-48c5ec8b3f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:39.944049 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:39.944665 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:39.944940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:39.945250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] PATH_INFO: `/v3/users/6c876432dba9499da480c151af4fe6ca` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:39.946378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c876432dba9499da480c151af4fe6ca)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:39.948291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:39.959080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ba4e71ad-442c-4826-b77f-48c5ec8b3f98 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:39.960294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ba4e71ad-442c-4826-b77f-48c5ec8b3f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:39.968396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 896/1776] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:39.977000 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8f679-f04d-4b90-9446-4f6ea9064a84 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:39.977347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8f679-f04d-4b90-9446-4f6ea9064a84 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:39.977747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8f679-f04d-4b90-9446-4f6ea9064a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:39.996798 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c876432dba9499da480c151af4fe6ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:39.998705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-47c8f679-f04d-4b90-9446-4f6ea9064a84 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:39.999900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-47c8f679-f04d-4b90-9446-4f6ea9064a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:40.006534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c876432dba9499da480c151af4fe6ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:40.007331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c876432dba9499da480c151af4fe6ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:40.007988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c876432dba9499da480c151af4fe6ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:40.009312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 897/1777] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:40.013556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8f679-f04d-4b90-9446-4f6ea9064a84 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.016997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:40.022874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c876432dba9499da480c151af4fe6ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:40.023027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8f679-f04d-4b90-9446-4f6ea9064a84 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:40.028130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22142b17-98e3-4665-997c-9aeea823c4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:40.028725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22142b17-98e3-4665-997c-9aeea823c4af admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.029001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22142b17-98e3-4665-997c-9aeea823c4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.029303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22142b17-98e3-4665-997c-9aeea823c4af admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.029935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22142b17-98e3-4665-997c-9aeea823c4af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:40.030231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22142b17-98e3-4665-997c-9aeea823c4af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:40.032663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22142b17-98e3-4665-997c-9aeea823c4af admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:40.039682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 881/1778] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:39 2022] DELETE /identity/v3/users/6c876432dba9499da480c151af4fe6ca => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:40.049979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 898/1779] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:40.052040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.054090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:40.139234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841a405-8d7e-4da2-bb89-2add4dba2367 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:40.157485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080c69-feec-4892-a7d0-a6b481aa610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:40.158320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080c69-feec-4892-a7d0-a6b481aa610f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.158534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080c69-feec-4892-a7d0-a6b481aa610f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.158941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080c69-feec-4892-a7d0-a6b481aa610f admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.159765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080c69-feec-4892-a7d0-a6b481aa610f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:40.161700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080c69-feec-4892-a7d0-a6b481aa610f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:40.202268 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 882/1780] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:40 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:01:40.202535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22142b17-98e3-4665-997c-9aeea823c4af admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.220681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22142b17-98e3-4665-997c-9aeea823c4af admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:40.301640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509a8f9-14fa-480b-a57d-24dbbafa003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:40.302414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509a8f9-14fa-480b-a57d-24dbbafa003a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.302914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509a8f9-14fa-480b-a57d-24dbbafa003a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.303158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509a8f9-14fa-480b-a57d-24dbbafa003a admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.304007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509a8f9-14fa-480b-a57d-24dbbafa003a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:40.306038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509a8f9-14fa-480b-a57d-24dbbafa003a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:40.315207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:40.332561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 899/1781] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:40 2022] POST /identity/v3/users => generated 283 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:01:40.336746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080c69-feec-4892-a7d0-a6b481aa610f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.348936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080c69-feec-4892-a7d0-a6b481aa610f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:40.356230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bb62d-5165-4c81-a05f-6fc45cad203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:40.356853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bb62d-5165-4c81-a05f-6fc45cad203d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.357165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bb62d-5165-4c81-a05f-6fc45cad203d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.357429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bb62d-5165-4c81-a05f-6fc45cad203d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.358098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bb62d-5165-4c81-a05f-6fc45cad203d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:40.360231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bb62d-5165-4c81-a05f-6fc45cad203d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:40.371529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 883/1782] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:40.376060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509a8f9-14fa-480b-a57d-24dbbafa003a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.386717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509a8f9-14fa-480b-a57d-24dbbafa003a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:40.391294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cce4b2-3ed8-405e-a3e8-9bc2f0934a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:40.391887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cce4b2-3ed8-405e-a3e8-9bc2f0934a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.392954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cce4b2-3ed8-405e-a3e8-9bc2f0934a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.393565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cce4b2-3ed8-405e-a3e8-9bc2f0934a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.394224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cce4b2-3ed8-405e-a3e8-9bc2f0934a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:40.395819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cce4b2-3ed8-405e-a3e8-9bc2f0934a21 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:40.409315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 900/1783] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:40.413172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bb62d-5165-4c81-a05f-6fc45cad203d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.424692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bb62d-5165-4c81-a05f-6fc45cad203d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:40.429487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068d49f-4b78-49c4-b56b-aa4d1dd1250d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:40.430133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068d49f-4b78-49c4-b56b-aa4d1dd1250d admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.430407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068d49f-4b78-49c4-b56b-aa4d1dd1250d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.430674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068d49f-4b78-49c4-b56b-aa4d1dd1250d admin admin] PATH_INFO: `/v3/projects/b6df348a7643492b86538f73919c5752/users/290bd0cdcb764fdc8bdb4011e2766991/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.435674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cce4b2-3ed8-405e-a3e8-9bc2f0934a21 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.437500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b9cce4b2-3ed8-405e-a3e8-9bc2f0934a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:40.451520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068d49f-4b78-49c4-b56b-aa4d1dd1250d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6df348a7643492b86538f73919c5752, user_id=290bd0cdcb764fdc8bdb4011e2766991, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:40.455320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068d49f-4b78-49c4-b56b-aa4d1dd1250d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:40.463719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cce4b2-3ed8-405e-a3e8-9bc2f0934a21 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:40.472403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 884/1784] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:40 2022] PUT /identity/v3/projects/b6df348a7643492b86538f73919c5752/users/290bd0cdcb764fdc8bdb4011e2766991/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:40.486328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eee6d3-8b68-41e9-9acd-33e2400acc3f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.486328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eee6d3-8b68-41e9-9acd-33e2400acc3f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.486328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eee6d3-8b68-41e9-9acd-33e2400acc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.519491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-56eee6d3-8b68-41e9-9acd-33e2400acc3f None None] MFA Rules not processed for user `290bd0cdcb764fdc8bdb4011e2766991`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:40.520826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-56eee6d3-8b68-41e9-9acd-33e2400acc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:40.560676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:40.562315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.562602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.564360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] PATH_INFO: `/v3/projects/116ffc7fd32f41c3bdf8218e3a8de537` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.565613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=116ffc7fd32f41c3bdf8218e3a8de537)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:40.567536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:40.606815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 885/1785] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:40.608121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eee6d3-8b68-41e9-9acd-33e2400acc3f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.627793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eee6d3-8b68-41e9-9acd-33e2400acc3f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:40.630204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116ffc7fd32f41c3bdf8218e3a8de537', 'request_id': 'req-e2139986-a25b-4b35-b23a-1619c643eb6d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:40.654839 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116ffc7fd32f41c3bdf8218e3a8de537', 'request_id': 'req-e2139986-a25b-4b35-b23a-1619c643eb6d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:40.662223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:40.677631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116ffc7fd32f41c3bdf8218e3a8de537', 'request_id': 'req-e2139986-a25b-4b35-b23a-1619c643eb6d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:40.699605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] The token cache is being invalidate because project 116ffc7fd32f41c3bdf8218e3a8de537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:40.699893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:40.701392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e2139986-a25b-4b35-b23a-1619c643eb6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:40.704469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 901/1786] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:40 2022] DELETE /identity/v3/projects/116ffc7fd32f41c3bdf8218e3a8de537 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:40.708781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebef020-ea1f-41a2-a45f-3748044eb8a1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.709093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebef020-ea1f-41a2-a45f-3748044eb8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.709810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebef020-ea1f-41a2-a45f-3748044eb8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.740064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fb903-6ff6-4617-b660-c61e8f6300f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:40.740877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fb903-6ff6-4617-b660-c61e8f6300f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.741309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fb903-6ff6-4617-b660-c61e8f6300f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.742728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fb903-6ff6-4617-b660-c61e8f6300f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.742728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fb903-6ff6-4617-b660-c61e8f6300f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:40.744326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fb903-6ff6-4617-b660-c61e8f6300f2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:40.762701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1ebef020-ea1f-41a2-a45f-3748044eb8a1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:40.764715 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1ebef020-ea1f-41a2-a45f-3748044eb8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:40.785137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 902/1787] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:40.789721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c7374-7531-4ab8-b40d-27c7a52fda15 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.790188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c7374-7531-4ab8-b40d-27c7a52fda15 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.790596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c7374-7531-4ab8-b40d-27c7a52fda15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.818234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-962c7374-7531-4ab8-b40d-27c7a52fda15 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:40.818473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 886/1788] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:40 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:40.819700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fb903-6ff6-4617-b660-c61e8f6300f2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.820500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-962c7374-7531-4ab8-b40d-27c7a52fda15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:40.827054 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-143fb903-6ff6-4617-b660-c61e8f6300f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:40.964404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fb903-6ff6-4617-b660-c61e8f6300f2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:40.979631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4dcc2-d663-4f7e-984c-52e3ad379384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:40.980465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4dcc2-d663-4f7e-984c-52e3ad379384 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:40.980997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4dcc2-d663-4f7e-984c-52e3ad379384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:40.981204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4dcc2-d663-4f7e-984c-52e3ad379384 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:40.981865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4dcc2-d663-4f7e-984c-52e3ad379384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:40.983781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4dcc2-d663-4f7e-984c-52e3ad379384 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:40.987956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 903/1789] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:40.992233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c7374-7531-4ab8-b40d-27c7a52fda15 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:40.996982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:41.006592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c7374-7531-4ab8-b40d-27c7a52fda15 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.017583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0262c2-95a5-4b2f-95bb-9f3b90b13a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.018305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0262c2-95a5-4b2f-95bb-9f3b90b13a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.018494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0262c2-95a5-4b2f-95bb-9f3b90b13a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.018768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0262c2-95a5-4b2f-95bb-9f3b90b13a7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.019421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0262c2-95a5-4b2f-95bb-9f3b90b13a7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:41.019745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0262c2-95a5-4b2f-95bb-9f3b90b13a7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.021392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0262c2-95a5-4b2f-95bb-9f3b90b13a7b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.034607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 887/1790] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:40 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:41.035586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4dcc2-d663-4f7e-984c-52e3ad379384 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.046507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 904/1791] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:41.051286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4dcc2-d663-4f7e-984c-52e3ad379384 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.052689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0262c2-95a5-4b2f-95bb-9f3b90b13a7b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.070925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272308b-024f-4bb2-8ccd-4d38804b6604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.071600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272308b-024f-4bb2-8ccd-4d38804b6604 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.071880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272308b-024f-4bb2-8ccd-4d38804b6604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.072607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272308b-024f-4bb2-8ccd-4d38804b6604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.073295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272308b-024f-4bb2-8ccd-4d38804b6604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.075170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272308b-024f-4bb2-8ccd-4d38804b6604 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.080272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0262c2-95a5-4b2f-95bb-9f3b90b13a7b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.086115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77a188-fede-4bfd-b2af-432893f5bb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.087228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77a188-fede-4bfd-b2af-432893f5bb2d admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.087503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77a188-fede-4bfd-b2af-432893f5bb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.087865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77a188-fede-4bfd-b2af-432893f5bb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.088867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77a188-fede-4bfd-b2af-432893f5bb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.091564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77a188-fede-4bfd-b2af-432893f5bb2d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.115558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 888/1792] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:41.119873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272308b-024f-4bb2-8ccd-4d38804b6604 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.155044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272308b-024f-4bb2-8ccd-4d38804b6604 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.169191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 905/1793] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:41 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:41.170704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77a188-fede-4bfd-b2af-432893f5bb2d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.298396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77a188-fede-4bfd-b2af-432893f5bb2d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.312994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866e02a-f7c9-4656-bd08-21a2d3e8f2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.314702 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12dbdc-d65f-45e4-8d5f-bc0d3026d408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.315574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866e02a-f7c9-4656-bd08-21a2d3e8f2e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.316432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866e02a-f7c9-4656-bd08-21a2d3e8f2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.316922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12dbdc-d65f-45e4-8d5f-bc0d3026d408 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.317305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866e02a-f7c9-4656-bd08-21a2d3e8f2e9 admin admin] PATH_INFO: `/v3/projects/d14c5e9598944023a2b392e5351fd068/users/52b0a8de8a6240a3aa94f77df56e108e/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.319216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12dbdc-d65f-45e4-8d5f-bc0d3026d408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.319216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12dbdc-d65f-45e4-8d5f-bc0d3026d408 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.319216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12dbdc-d65f-45e4-8d5f-bc0d3026d408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.325557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12dbdc-d65f-45e4-8d5f-bc0d3026d408 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.356314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866e02a-f7c9-4656-bd08-21a2d3e8f2e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14c5e9598944023a2b392e5351fd068, user_id=52b0a8de8a6240a3aa94f77df56e108e, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.360899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866e02a-f7c9-4656-bd08-21a2d3e8f2e9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.373225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 906/1794] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:41 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:41.378312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12dbdc-d65f-45e4-8d5f-bc0d3026d408 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.383292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 889/1795] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:41 2022] PUT /identity/v3/projects/d14c5e9598944023a2b392e5351fd068/users/52b0a8de8a6240a3aa94f77df56e108e/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:41.390036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866e02a-f7c9-4656-bd08-21a2d3e8f2e9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.400960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12dbdc-d65f-45e4-8d5f-bc0d3026d408 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.419237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:41.490190 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866e02a-f7c9-4656-bd08-21a2d3e8f2e9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.497025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abbde3-534e-450e-b1fe-a88ac7b4d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.499072 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abbde3-534e-450e-b1fe-a88ac7b4d57d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.504646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abbde3-534e-450e-b1fe-a88ac7b4d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.505989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abbde3-534e-450e-b1fe-a88ac7b4d57d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.506794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abbde3-534e-450e-b1fe-a88ac7b4d57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.509275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abbde3-534e-450e-b1fe-a88ac7b4d57d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.526866 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 890/1796] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:41.528493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abbde3-534e-450e-b1fe-a88ac7b4d57d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.549019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abbde3-534e-450e-b1fe-a88ac7b4d57d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.552095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7225a47-8941-47bb-97ff-3299b206fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.552741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7225a47-8941-47bb-97ff-3299b206fc78 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.553129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7225a47-8941-47bb-97ff-3299b206fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.553329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7225a47-8941-47bb-97ff-3299b206fc78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.554050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7225a47-8941-47bb-97ff-3299b206fc78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.559775 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7225a47-8941-47bb-97ff-3299b206fc78 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.568361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fbf1b-cf0c-4038-8622-de4d5bb207f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.569353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fbf1b-cf0c-4038-8622-de4d5bb207f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.569738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fbf1b-cf0c-4038-8622-de4d5bb207f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.569919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fbf1b-cf0c-4038-8622-de4d5bb207f8 admin admin] PATH_INFO: `/v3/projects/d14c5e9598944023a2b392e5351fd068/users/52b0a8de8a6240a3aa94f77df56e108e/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.572441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 907/1797] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:41.576072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7225a47-8941-47bb-97ff-3299b206fc78 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.581669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fbf1b-cf0c-4038-8622-de4d5bb207f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14c5e9598944023a2b392e5351fd068, user_id=52b0a8de8a6240a3aa94f77df56e108e, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.584374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fbf1b-cf0c-4038-8622-de4d5bb207f8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.600234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 891/1798] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:41 2022] PUT /identity/v3/projects/d14c5e9598944023a2b392e5351fd068/users/52b0a8de8a6240a3aa94f77df56e108e/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:41.614599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce79fbe-c54e-4ac0-8fe1-6340201ae9d9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.615908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce79fbe-c54e-4ac0-8fe1-6340201ae9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.616384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce79fbe-c54e-4ac0-8fe1-6340201ae9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.617721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7225a47-8941-47bb-97ff-3299b206fc78 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.658187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2ce79fbe-c54e-4ac0-8fe1-6340201ae9d9 None None] MFA Rules not processed for user `52b0a8de8a6240a3aa94f77df56e108e`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:41.659875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2ce79fbe-c54e-4ac0-8fe1-6340201ae9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:41.684604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:41.700505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd41268-92d1-4f43-bcd4-81cae756dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.701542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd41268-92d1-4f43-bcd4-81cae756dfbd admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.701982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd41268-92d1-4f43-bcd4-81cae756dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.702404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd41268-92d1-4f43-bcd4-81cae756dfbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.703298 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd41268-92d1-4f43-bcd4-81cae756dfbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.706997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd41268-92d1-4f43-bcd4-81cae756dfbd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.733190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 908/1799] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:41.739396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd41268-92d1-4f43-bcd4-81cae756dfbd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.754919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 892/1800] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:41.768560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd41268-92d1-4f43-bcd4-81cae756dfbd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.770122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce79fbe-c54e-4ac0-8fe1-6340201ae9d9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:41.772050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2ce79fbe-c54e-4ac0-8fe1-6340201ae9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:41.784277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c8012-f0d0-44e1-8e87-b7079a0c73d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.785307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c8012-f0d0-44e1-8e87-b7079a0c73d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.785593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c8012-f0d0-44e1-8e87-b7079a0c73d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.786071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c8012-f0d0-44e1-8e87-b7079a0c73d1 admin admin] PATH_INFO: `/v3/projects/297933af06fe49d99834d31c4aeb193e/users/2db27aea5c6640e4b6235897589c14d6/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.828655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c8012-f0d0-44e1-8e87-b7079a0c73d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297933af06fe49d99834d31c4aeb193e, user_id=2db27aea5c6640e4b6235897589c14d6, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:41.835582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c8012-f0d0-44e1-8e87-b7079a0c73d1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:41.859080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce79fbe-c54e-4ac0-8fe1-6340201ae9d9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:41.881172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 909/1801] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:41 2022] PUT /identity/v3/projects/297933af06fe49d99834d31c4aeb193e/users/2db27aea5c6640e4b6235897589c14d6/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:41.892742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b699a-5e17-4ccb-a7fa-bbb0339ba1e2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.893031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b699a-5e17-4ccb-a7fa-bbb0339ba1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.893334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b699a-5e17-4ccb-a7fa-bbb0339ba1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:41.926361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:41.943220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-df6b699a-5e17-4ccb-a7fa-bbb0339ba1e2 None None] MFA Rules not processed for user `2db27aea5c6640e4b6235897589c14d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:41.944663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-df6b699a-5e17-4ccb-a7fa-bbb0339ba1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:41.968049 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf3312-9338-4b37-81c9-c8dcef02fc95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:41.968821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf3312-9338-4b37-81c9-c8dcef02fc95 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:41.970646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf3312-9338-4b37-81c9-c8dcef02fc95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:41.970913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf3312-9338-4b37-81c9-c8dcef02fc95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:42.062478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf3312-9338-4b37-81c9-c8dcef02fc95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:42.065421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf3312-9338-4b37-81c9-c8dcef02fc95 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:42.073897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 910/1802] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:42.078326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b699a-5e17-4ccb-a7fa-bbb0339ba1e2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:42.089130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 893/1803] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:42.207850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b699a-5e17-4ccb-a7fa-bbb0339ba1e2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:42.214302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3450c8-5580-4250-a300-00f977eae452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:42.215294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3450c8-5580-4250-a300-00f977eae452 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:42.220642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3450c8-5580-4250-a300-00f977eae452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:42.220942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3450c8-5580-4250-a300-00f977eae452 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:42.221646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3450c8-5580-4250-a300-00f977eae452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:42.223574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3450c8-5580-4250-a300-00f977eae452 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:42.275090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 911/1804] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:42 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:42.275221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf3312-9338-4b37-81c9-c8dcef02fc95 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:42.306625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6eefc2-4c1f-4222-b698-e0b93bdc34ff None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:42.306909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6eefc2-4c1f-4222-b698-e0b93bdc34ff None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:42.326653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6eefc2-4c1f-4222-b698-e0b93bdc34ff None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:42.327434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf3312-9338-4b37-81c9-c8dcef02fc95 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:42.334972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 912/1805] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:01:42 2022] GET /identity => generated 270 bytes in 32 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:42.342791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018c49b-bac2-4cc3-8e08-624f7f30abbb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:42.343074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018c49b-bac2-4cc3-8e08-624f7f30abbb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:42.352230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018c49b-bac2-4cc3-8e08-624f7f30abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:42.442682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a018c49b-bac2-4cc3-8e08-624f7f30abbb None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:42.448003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a018c49b-bac2-4cc3-8e08-624f7f30abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:42.584582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:42.595981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4332bd-4aff-437e-ae5a-883916be866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:42.599842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4332bd-4aff-437e-ae5a-883916be866f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:42.599842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4332bd-4aff-437e-ae5a-883916be866f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:42.599842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4332bd-4aff-437e-ae5a-883916be866f admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:42.602794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4332bd-4aff-437e-ae5a-883916be866f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:42.608379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4332bd-4aff-437e-ae5a-883916be866f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:42.646430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 894/1806] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:42 2022] POST /identity/v3/users => generated 298 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:42.646430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4332bd-4aff-437e-ae5a-883916be866f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:42.649963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0a4332bd-4aff-437e-ae5a-883916be866f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:42.666322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 913/1807] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:01:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:42.667202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018c49b-bac2-4cc3-8e08-624f7f30abbb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:42.681321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4332bd-4aff-437e-ae5a-883916be866f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:42.693824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018c49b-bac2-4cc3-8e08-624f7f30abbb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:42.694589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:42.695238 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:42.695514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:42.695783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] PATH_INFO: `/v3/users/3f90d1ff80064806aa7b760f1fbca680` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:42.697295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f90d1ff80064806aa7b760f1fbca680)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:42.699216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:42.705693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dedede-7e71-4f93-bc40-9560b8b7f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:42.706338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dedede-7e71-4f93-bc40-9560b8b7f848 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:42.706583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dedede-7e71-4f93-bc40-9560b8b7f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:42.706849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dedede-7e71-4f93-bc40-9560b8b7f848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:42.707549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dedede-7e71-4f93-bc40-9560b8b7f848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:42.711564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dedede-7e71-4f93-bc40-9560b8b7f848 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:42.741774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 914/1808] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:42.749131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dedede-7e71-4f93-bc40-9560b8b7f848 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:42.767849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dedede-7e71-4f93-bc40-9560b8b7f848 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:42.778705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7997f-1aa3-4342-9f1b-cb9cf07f0d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:42.779279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7997f-1aa3-4342-9f1b-cb9cf07f0d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:42.779559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7997f-1aa3-4342-9f1b-cb9cf07f0d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:42.779895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7997f-1aa3-4342-9f1b-cb9cf07f0d58 admin admin] PATH_INFO: `/v3/projects/5f21a70ce0e14ca8bff3e3109d00e70e/users/42b76617c18649b1afb0d85fc1df11ad/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:42.817958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7997f-1aa3-4342-9f1b-cb9cf07f0d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f21a70ce0e14ca8bff3e3109d00e70e, user_id=42b76617c18649b1afb0d85fc1df11ad, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:42.820622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7997f-1aa3-4342-9f1b-cb9cf07f0d58 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:42.829717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f90d1ff80064806aa7b760f1fbca680'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:42.846943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f90d1ff80064806aa7b760f1fbca680'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:42.848504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f90d1ff80064806aa7b760f1fbca680'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:42.849159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f90d1ff80064806aa7b760f1fbca680'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:42.852460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 915/1809] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:42 2022] PUT /identity/v3/projects/5f21a70ce0e14ca8bff3e3109d00e70e/users/42b76617c18649b1afb0d85fc1df11ad/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:42.852460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7997f-1aa3-4342-9f1b-cb9cf07f0d58 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:42.854432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a2f7997f-1aa3-4342-9f1b-cb9cf07f0d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:42.868568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:42.871638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f90d1ff80064806aa7b760f1fbca680'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:42.907619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 895/1810] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:42 2022] DELETE /identity/v3/users/3f90d1ff80064806aa7b760f1fbca680 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:42.908942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:42.913710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.063578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38091f41-a02f-471a-ac0c-017ed6f02e77 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:43.071476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6b2d6-81d0-4f27-b320-94ba78b89212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:43.072199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6b2d6-81d0-4f27-b320-94ba78b89212 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.072554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6b2d6-81d0-4f27-b320-94ba78b89212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.072979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6b2d6-81d0-4f27-b320-94ba78b89212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.074278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6b2d6-81d0-4f27-b320-94ba78b89212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:43.076209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6b2d6-81d0-4f27-b320-94ba78b89212 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:43.076517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7997f-1aa3-4342-9f1b-cb9cf07f0d58 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:43.096472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 896/1811] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:43.100614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fb19c-da5c-476f-b478-13ae7d72f666 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.100992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fb19c-da5c-476f-b478-13ae7d72f666 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.102750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fb19c-da5c-476f-b478-13ae7d72f666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.151693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c80fb19c-da5c-476f-b478-13ae7d72f666 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:43.153752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c80fb19c-da5c-476f-b478-13ae7d72f666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.176727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 897/1812] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:43.176727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fb19c-da5c-476f-b478-13ae7d72f666 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:43.177508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c80fb19c-da5c-476f-b478-13ae7d72f666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.194349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fb19c-da5c-476f-b478-13ae7d72f666 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:43.202182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:43.204131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.204952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.205414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] PATH_INFO: `/v3/users/290bd0cdcb764fdc8bdb4011e2766991` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.207495 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] RBAC: Authorizing `identity:delete_user(user_id=290bd0cdcb764fdc8bdb4011e2766991)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:43.211121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:43.224499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69cf67-4553-448e-aa65-d48bce97a227 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:43.226427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69cf67-4553-448e-aa65-d48bce97a227 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.227115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69cf67-4553-448e-aa65-d48bce97a227 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.228142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69cf67-4553-448e-aa65-d48bce97a227 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.232413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-aa69cf67-4553-448e-aa65-d48bce97a227 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.249252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69cf67-4553-448e-aa65-d48bce97a227 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:43.253590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69cf67-4553-448e-aa65-d48bce97a227 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:43.276025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 916/1813] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:43.277644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69cf67-4553-448e-aa65-d48bce97a227 service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:43.306134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69cf67-4553-448e-aa65-d48bce97a227 service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:43.316686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97601f5-6513-4a5d-90c5-07692c796e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:43.317837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97601f5-6513-4a5d-90c5-07692c796e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.318926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97601f5-6513-4a5d-90c5-07692c796e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.319426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97601f5-6513-4a5d-90c5-07692c796e59 admin admin] PATH_INFO: `/v3/projects/5f21a70ce0e14ca8bff3e3109d00e70e/users/42b76617c18649b1afb0d85fc1df11ad/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.323536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97601f5-6513-4a5d-90c5-07692c796e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f21a70ce0e14ca8bff3e3109d00e70e, user_id=42b76617c18649b1afb0d85fc1df11ad, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:43.330810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97601f5-6513-4a5d-90c5-07692c796e59 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:43.333651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290bd0cdcb764fdc8bdb4011e2766991'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:43.349401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290bd0cdcb764fdc8bdb4011e2766991'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:43.350098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290bd0cdcb764fdc8bdb4011e2766991'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:43.350734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290bd0cdcb764fdc8bdb4011e2766991'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:43.377559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ead78ec-3ccf-4392-9020-6ae8a3d4a05c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290bd0cdcb764fdc8bdb4011e2766991'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:43.382036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 917/1814] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:43 2022] PUT /identity/v3/projects/5f21a70ce0e14ca8bff3e3109d00e70e/users/42b76617c18649b1afb0d85fc1df11ad/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:43.392174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a3dbc-97cf-4067-b715-90a7778833f7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.392978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a3dbc-97cf-4067-b715-90a7778833f7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.393673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a3dbc-97cf-4067-b715-90a7778833f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.418529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 898/1815] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:43 2022] DELETE /identity/v3/users/290bd0cdcb764fdc8bdb4011e2766991 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:43.420088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50907177-130e-456f-a71b-364b236e16fb None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.420380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50907177-130e-456f-a71b-364b236e16fb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.420654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50907177-130e-456f-a71b-364b236e16fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.436747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:43.460498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-50907177-130e-456f-a71b-364b236e16fb None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:43.461814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-50907177-130e-456f-a71b-364b236e16fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.473321 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7d7a3dbc-97cf-4067-b715-90a7778833f7 None None] MFA Rules not processed for user `42b76617c18649b1afb0d85fc1df11ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:43.475509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7d7a3dbc-97cf-4067-b715-90a7778833f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.520182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:43.554510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 899/1816] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:43.556327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50907177-130e-456f-a71b-364b236e16fb None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:43.558413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-50907177-130e-456f-a71b-364b236e16fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.605847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 918/1817] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:43.619169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a3dbc-97cf-4067-b715-90a7778833f7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:43.620729 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7d7a3dbc-97cf-4067-b715-90a7778833f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.701937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50907177-130e-456f-a71b-364b236e16fb None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:43.716614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9781-7f93-4528-84e5-31bb6c53f77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:43.717331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9781-7f93-4528-84e5-31bb6c53f77d service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.717611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9781-7f93-4528-84e5-31bb6c53f77d service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.717912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9781-7f93-4528-84e5-31bb6c53f77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.721112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d93f9781-7f93-4528-84e5-31bb6c53f77d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.722805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a3dbc-97cf-4067-b715-90a7778833f7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:43.727059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a81bd4-fdf1-46ce-92fc-f3c23ff700ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:43.727851 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a81bd4-fdf1-46ce-92fc-f3c23ff700ce service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.728412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a81bd4-fdf1-46ce-92fc-f3c23ff700ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.728864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a81bd4-fdf1-46ce-92fc-f3c23ff700ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.744252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a81bd4-fdf1-46ce-92fc-f3c23ff700ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:43.746079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a81bd4-fdf1-46ce-92fc-f3c23ff700ce service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:43.755290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9781-7f93-4528-84e5-31bb6c53f77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:43.760640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9781-7f93-4528-84e5-31bb6c53f77d service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:43.772815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 919/1818] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:43.776156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a81bd4-fdf1-46ce-92fc-f3c23ff700ce service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:43.780063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-51a81bd4-fdf1-46ce-92fc-f3c23ff700ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:43.798099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 900/1819] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:43.821908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a81bd4-fdf1-46ce-92fc-f3c23ff700ce service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:43.832106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c6372-d01e-46b7-82f2-e824ccb9499e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:43.832925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c6372-d01e-46b7-82f2-e824ccb9499e service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.833213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c6372-d01e-46b7-82f2-e824ccb9499e service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.833558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c6372-d01e-46b7-82f2-e824ccb9499e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.851399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c6372-d01e-46b7-82f2-e824ccb9499e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:43.853352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c6372-d01e-46b7-82f2-e824ccb9499e service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:43.874525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 920/1820] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:43.905475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9781-7f93-4528-84e5-31bb6c53f77d service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:43.917416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9781-7f93-4528-84e5-31bb6c53f77d service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:43.922925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd156f6-f853-4628-b598-57d0ad6f1a20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:43.923856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd156f6-f853-4628-b598-57d0ad6f1a20 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:43.924237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd156f6-f853-4628-b598-57d0ad6f1a20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:43.924610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd156f6-f853-4628-b598-57d0ad6f1a20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:43.926929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8dd156f6-f853-4628-b598-57d0ad6f1a20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:44.002510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd156f6-f853-4628-b598-57d0ad6f1a20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:44.004307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd156f6-f853-4628-b598-57d0ad6f1a20 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:44.018472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 901/1821] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:01:44.321064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c6372-d01e-46b7-82f2-e824ccb9499e service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:44.322995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-265c6372-d01e-46b7-82f2-e824ccb9499e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:44.348398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c6372-d01e-46b7-82f2-e824ccb9499e service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:44.355961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:44.356781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:44.357453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:44.357834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] PATH_INFO: `/v3/projects/b6df348a7643492b86538f73919c5752` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:44.359123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6df348a7643492b86538f73919c5752)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:44.362721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:44.449725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6df348a7643492b86538f73919c5752', 'request_id': 'req-a6408ddd-4a70-46f0-a3be-5bb47437a073'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.485203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6df348a7643492b86538f73919c5752', 'request_id': 'req-a6408ddd-4a70-46f0-a3be-5bb47437a073'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.485203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6df348a7643492b86538f73919c5752', 'request_id': 'req-a6408ddd-4a70-46f0-a3be-5bb47437a073'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.511325 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] The token cache is being invalidate because project b6df348a7643492b86538f73919c5752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:44.511822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.512831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.515024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 921/1822] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:44 2022] DELETE /identity/v3/projects/b6df348a7643492b86538f73919c5752 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:44.519683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd156f6-f853-4628-b598-57d0ad6f1a20 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:44.534157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd156f6-f853-4628-b598-57d0ad6f1a20 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:44.618116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:44.618947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:44.619321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:44.619781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] PATH_INFO: `/v3/users/52b0a8de8a6240a3aa94f77df56e108e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:44.622474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:44.632548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:44.643040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52b0a8de8a6240a3aa94f77df56e108e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:44.647277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:44.651093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:44.660971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6408ddd-4a70-46f0-a3be-5bb47437a073 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:44.676201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:44.676902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:44.677287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:44.677470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] PATH_INFO: `/v3/projects/7951cc4f90214786a438cc94e80ace18` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:44.678635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7951cc4f90214786a438cc94e80ace18)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:44.681538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:44.711221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:44.733277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52b0a8de8a6240a3aa94f77df56e108e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.749421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52b0a8de8a6240a3aa94f77df56e108e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.754379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52b0a8de8a6240a3aa94f77df56e108e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.754810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52b0a8de8a6240a3aa94f77df56e108e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.758184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7951cc4f90214786a438cc94e80ace18', 'request_id': 'req-0f597559-93df-4a4e-b77e-3bac0fbbd560'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.767648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7951cc4f90214786a438cc94e80ace18', 'request_id': 'req-0f597559-93df-4a4e-b77e-3bac0fbbd560'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.785577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52b0a8de8a6240a3aa94f77df56e108e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.791256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7951cc4f90214786a438cc94e80ace18', 'request_id': 'req-0f597559-93df-4a4e-b77e-3bac0fbbd560'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.815495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] The token cache is being invalidate because project 7951cc4f90214786a438cc94e80ace18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:44.815921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.816701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:44.822717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 922/1823] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:44 2022] DELETE /identity/v3/projects/7951cc4f90214786a438cc94e80ace18 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:44.824732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 902/1824] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:44 2022] DELETE /identity/v3/users/52b0a8de8a6240a3aa94f77df56e108e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:44.826995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:44.830200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:44.951312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f597559-93df-4a4e-b77e-3bac0fbbd560 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:45.076857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:45.078149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:45.078756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:45.079306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] PATH_INFO: `/v3/users/532a47038bac49b7b4264860421bd534` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:45.081742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=532a47038bac49b7b4264860421bd534)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:45.085222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:45.161680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:45.188952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532a47038bac49b7b4264860421bd534'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.214926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532a47038bac49b7b4264860421bd534'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.227006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532a47038bac49b7b4264860421bd534'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.228007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532a47038bac49b7b4264860421bd534'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.249119 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532a47038bac49b7b4264860421bd534'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.261342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 923/1825] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:44 2022] DELETE /identity/v3/users/532a47038bac49b7b4264860421bd534 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:45.674433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:45.675913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:45.700377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b789b-725f-40e4-a276-cabc93082f49 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:45.781264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:45.782598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:45.782598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:45.782786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] PATH_INFO: `/v3/projects/d14c5e9598944023a2b392e5351fd068` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:45.784036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14c5e9598944023a2b392e5351fd068)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:45.787319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:45.808035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:45.810143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:45.836578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ef7c0-f5a1-4ff3-b0de-f73dc894eb3d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:45.867644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:45.871585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:45.871979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:45.873572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] PATH_INFO: `/v3/projects/20dbde44f7d84f87b124c48bb9d2fdaa` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:45.876107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20dbde44f7d84f87b124c48bb9d2fdaa)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:45.888706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:45.893514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14c5e9598944023a2b392e5351fd068', 'request_id': 'req-aa45bf7e-6367-436b-ad0f-8024807c8786'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.905866 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14c5e9598944023a2b392e5351fd068', 'request_id': 'req-aa45bf7e-6367-436b-ad0f-8024807c8786'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.920982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14c5e9598944023a2b392e5351fd068', 'request_id': 'req-aa45bf7e-6367-436b-ad0f-8024807c8786'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.958457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] The token cache is being invalidate because project d14c5e9598944023a2b392e5351fd068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:45.958763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.959582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa45bf7e-6367-436b-ad0f-8024807c8786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.962386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 903/1826] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:45 2022] DELETE /identity/v3/projects/d14c5e9598944023a2b392e5351fd068 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:45.974775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621b7ad-7dab-4983-b7e4-81b0a38fe1dc None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:45.975098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621b7ad-7dab-4983-b7e4-81b0a38fe1dc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:45.975645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621b7ad-7dab-4983-b7e4-81b0a38fe1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:45.979732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20dbde44f7d84f87b124c48bb9d2fdaa', 'request_id': 'req-e5e48d15-9d97-48c3-bedf-ce4db19bf772'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:45.988806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20dbde44f7d84f87b124c48bb9d2fdaa', 'request_id': 'req-e5e48d15-9d97-48c3-bedf-ce4db19bf772'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:46.005271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20dbde44f7d84f87b124c48bb9d2fdaa', 'request_id': 'req-e5e48d15-9d97-48c3-bedf-ce4db19bf772'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:46.046396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] The token cache is being invalidate because project 20dbde44f7d84f87b124c48bb9d2fdaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:46.046396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:46.046563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a621b7ad-7dab-4983-b7e4-81b0a38fe1dc None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:46.046652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e5e48d15-9d97-48c3-bedf-ce4db19bf772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:46.048864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a621b7ad-7dab-4983-b7e4-81b0a38fe1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:46.065744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 924/1827] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:45 2022] DELETE /identity/v3/projects/20dbde44f7d84f87b124c48bb9d2fdaa => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:46.071244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d5248-ad01-4e7b-a4d1-a9324c85feaa None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.071244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d5248-ad01-4e7b-a4d1-a9324c85feaa None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.071828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d5248-ad01-4e7b-a4d1-a9324c85feaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.145786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b55d5248-ad01-4e7b-a4d1-a9324c85feaa None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:46.147551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b55d5248-ad01-4e7b-a4d1-a9324c85feaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:46.182118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:46.301901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 904/1828] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:46.303622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc5c0b-6d37-4057-b03b-0be5197e7644 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.305532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc5c0b-6d37-4057-b03b-0be5197e7644 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.306010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc5c0b-6d37-4057-b03b-0be5197e7644 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.311083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 905/1829] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:01:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:46.313832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c887f2-fdc4-45ec-9cd7-1e7676a805da None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.315151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c887f2-fdc4-45ec-9cd7-1e7676a805da None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.315667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c887f2-fdc4-45ec-9cd7-1e7676a805da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.332364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 925/1830] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:46.339739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897039b5-df04-4431-b346-72e18a7e02ff None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.340066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897039b5-df04-4431-b346-72e18a7e02ff None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.340294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897039b5-df04-4431-b346-72e18a7e02ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.374662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-897039b5-df04-4431-b346-72e18a7e02ff None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:46.377290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-897039b5-df04-4431-b346-72e18a7e02ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:46.396347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-37c887f2-fdc4-45ec-9cd7-1e7676a805da None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:46.397629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-37c887f2-fdc4-45ec-9cd7-1e7676a805da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:46.427114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 906/1831] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:46.428238 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48790313-8dae-4c82-8131-796dd1a88b98 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.428518 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48790313-8dae-4c82-8131-796dd1a88b98 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.428789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48790313-8dae-4c82-8131-796dd1a88b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.461950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:46.475005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-48790313-8dae-4c82-8131-796dd1a88b98 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:46.486405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-48790313-8dae-4c82-8131-796dd1a88b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:46.509623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 907/1832] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:46.510980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48790313-8dae-4c82-8131-796dd1a88b98 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:46.545062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48790313-8dae-4c82-8131-796dd1a88b98 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:46.552397 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381611b-1f33-4a8d-8a28-672cdbd3b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:46.553023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381611b-1f33-4a8d-8a28-672cdbd3b42a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.553332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381611b-1f33-4a8d-8a28-672cdbd3b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.553602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381611b-1f33-4a8d-8a28-672cdbd3b42a admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.557730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381611b-1f33-4a8d-8a28-672cdbd3b42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:46.558047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381611b-1f33-4a8d-8a28-672cdbd3b42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:46.559707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381611b-1f33-4a8d-8a28-672cdbd3b42a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:46.573214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 926/1833] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:01:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:46.574378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897039b5-df04-4431-b346-72e18a7e02ff None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:46.589136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 908/1834] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:46.589136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381611b-1f33-4a8d-8a28-672cdbd3b42a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:46.608509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897039b5-df04-4431-b346-72e18a7e02ff None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:46.620569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6e739-e289-4fc2-9b71-ff75ed183c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:46.622263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6e739-e289-4fc2-9b71-ff75ed183c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.622564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6e739-e289-4fc2-9b71-ff75ed183c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.623181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6e739-e289-4fc2-9b71-ff75ed183c11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.623768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6e739-e289-4fc2-9b71-ff75ed183c11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:46.624206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6e739-e289-4fc2-9b71-ff75ed183c11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:46.627309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6e739-e289-4fc2-9b71-ff75ed183c11 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:46.638444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381611b-1f33-4a8d-8a28-672cdbd3b42a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:46.657807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbc278-583c-4aa7-aa54-f3171495268c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:46.658392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 927/1835] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:46.663226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6e739-e289-4fc2-9b71-ff75ed183c11 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:46.663874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbc278-583c-4aa7-aa54-f3171495268c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.665317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbc278-583c-4aa7-aa54-f3171495268c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.668260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbc278-583c-4aa7-aa54-f3171495268c admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.671626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbc278-583c-4aa7-aa54-f3171495268c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:46.679723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbc278-583c-4aa7-aa54-f3171495268c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:46.688734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6e739-e289-4fc2-9b71-ff75ed183c11 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:46.704429 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99619a31-ea65-4e86-9829-ece08a728c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:46.705057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99619a31-ea65-4e86-9829-ece08a728c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.705373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99619a31-ea65-4e86-9829-ece08a728c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.705703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99619a31-ea65-4e86-9829-ece08a728c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.706349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99619a31-ea65-4e86-9829-ece08a728c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:46.708300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99619a31-ea65-4e86-9829-ece08a728c00 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:46.766891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 909/1836] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:46 2022] POST /identity/v3/projects => generated 407 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:46.770979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbc278-583c-4aa7-aa54-f3171495268c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:46.808520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 928/1837] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:46 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:46.813981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99619a31-ea65-4e86-9829-ece08a728c00 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:46.974026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbc278-583c-4aa7-aa54-f3171495268c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:46.980781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99619a31-ea65-4e86-9829-ece08a728c00 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:46.986602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2ccfa-b8c6-4f84-953f-2279d403481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:46.987371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2ccfa-b8c6-4f84-953f-2279d403481d admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.988191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2ccfa-b8c6-4f84-953f-2279d403481d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.988955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2ccfa-b8c6-4f84-953f-2279d403481d admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.989988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2ccfa-b8c6-4f84-953f-2279d403481d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:46.992633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2ccfa-b8c6-4f84-953f-2279d403481d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:46.993466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896602d-08d5-4ef9-a82c-d90433ee69d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:46.994259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896602d-08d5-4ef9-a82c-d90433ee69d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:46.994594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896602d-08d5-4ef9-a82c-d90433ee69d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:46.997283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896602d-08d5-4ef9-a82c-d90433ee69d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:46.997989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896602d-08d5-4ef9-a82c-d90433ee69d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.000599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896602d-08d5-4ef9-a82c-d90433ee69d3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.037523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 910/1838] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:46 2022] POST /identity/v3/users => generated 304 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:01:47.041318 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2ccfa-b8c6-4f84-953f-2279d403481d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.043113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 929/1839] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:46 2022] POST /identity/v3/users => generated 282 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:47.052140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896602d-08d5-4ef9-a82c-d90433ee69d3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.055361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2ccfa-b8c6-4f84-953f-2279d403481d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.061134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d9874-f7a3-4f39-8f1b-8cb7b2f6ca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.061947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9874-f7a3-4f39-8f1b-8cb7b2f6ca20 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.062224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9874-f7a3-4f39-8f1b-8cb7b2f6ca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.063195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9874-f7a3-4f39-8f1b-8cb7b2f6ca20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.064704 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d9874-f7a3-4f39-8f1b-8cb7b2f6ca20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.066367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896602d-08d5-4ef9-a82c-d90433ee69d3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.066558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d9874-f7a3-4f39-8f1b-8cb7b2f6ca20 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.071166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d1905-c5a7-46fa-b254-2400c470d3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.071788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1905-c5a7-46fa-b254-2400c470d3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.072618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1905-c5a7-46fa-b254-2400c470d3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.072892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1905-c5a7-46fa-b254-2400c470d3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.073692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d1905-c5a7-46fa-b254-2400c470d3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.075267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d1905-c5a7-46fa-b254-2400c470d3b8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.081112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 911/1840] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:47.081918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d9874-f7a3-4f39-8f1b-8cb7b2f6ca20 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.093427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 930/1841] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:47.093602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d9874-f7a3-4f39-8f1b-8cb7b2f6ca20 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.095082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d1905-c5a7-46fa-b254-2400c470d3b8 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.097859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e308eca-cc9b-4414-9764-cd98af8d63dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.098675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e308eca-cc9b-4414-9764-cd98af8d63dc admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.099110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e308eca-cc9b-4414-9764-cd98af8d63dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.099483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e308eca-cc9b-4414-9764-cd98af8d63dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.100225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e308eca-cc9b-4414-9764-cd98af8d63dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.102916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e308eca-cc9b-4414-9764-cd98af8d63dc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.108603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d1905-c5a7-46fa-b254-2400c470d3b8 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.114651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 912/1842] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:01:47.117325 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1197956-4cc5-4e09-9add-60a11c4b7c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.117974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1197956-4cc5-4e09-9add-60a11c4b7c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.118256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1197956-4cc5-4e09-9add-60a11c4b7c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.118527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1197956-4cc5-4e09-9add-60a11c4b7c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.119177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1197956-4cc5-4e09-9add-60a11c4b7c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.120763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1197956-4cc5-4e09-9add-60a11c4b7c99 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.122266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e308eca-cc9b-4414-9764-cd98af8d63dc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.134197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e308eca-cc9b-4414-9764-cd98af8d63dc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.137005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 931/1843] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:01:47.141110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2844b-ffb8-4e92-b5b8-63e0e39a3186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.141611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1197956-4cc5-4e09-9add-60a11c4b7c99 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.142434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2844b-ffb8-4e92-b5b8-63e0e39a3186 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.143146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2844b-ffb8-4e92-b5b8-63e0e39a3186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.143146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2844b-ffb8-4e92-b5b8-63e0e39a3186 admin admin] PATH_INFO: `/v3/projects/0b22b91373e24bca8a52332a8d66d77c/users/457ea559dd084aec86d2c46e0baba76d/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.153817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1197956-4cc5-4e09-9add-60a11c4b7c99 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.159354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdd3c6-4aab-46f8-8386-3d4a90054945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.160140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdd3c6-4aab-46f8-8386-3d4a90054945 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.160638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdd3c6-4aab-46f8-8386-3d4a90054945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.161477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdd3c6-4aab-46f8-8386-3d4a90054945 admin admin] PATH_INFO: `/v3/projects/bfc85d8bc0fb463e97503dc2b3a20390/users/76961abde5d7401488285ac649ff5b61/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.175151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2844b-ffb8-4e92-b5b8-63e0e39a3186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b22b91373e24bca8a52332a8d66d77c, user_id=457ea559dd084aec86d2c46e0baba76d, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.177883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2844b-ffb8-4e92-b5b8-63e0e39a3186 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.180309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdd3c6-4aab-46f8-8386-3d4a90054945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc85d8bc0fb463e97503dc2b3a20390, user_id=76961abde5d7401488285ac649ff5b61, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.182935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdd3c6-4aab-46f8-8386-3d4a90054945 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.204115 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 913/1844] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:47 2022] PUT /identity/v3/projects/0b22b91373e24bca8a52332a8d66d77c/users/457ea559dd084aec86d2c46e0baba76d/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:47.205800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2844b-ffb8-4e92-b5b8-63e0e39a3186 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.207937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6ee2844b-ffb8-4e92-b5b8-63e0e39a3186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.213152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 932/1845] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:47 2022] PUT /identity/v3/projects/bfc85d8bc0fb463e97503dc2b3a20390/users/76961abde5d7401488285ac649ff5b61/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:47.214314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdd3c6-4aab-46f8-8386-3d4a90054945 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.216281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1dcdd3c6-4aab-46f8-8386-3d4a90054945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.288626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:47.309867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:47.331819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2844b-ffb8-4e92-b5b8-63e0e39a3186 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.353614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdd3c6-4aab-46f8-8386-3d4a90054945 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.363335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed3b75-04e3-4309-bac3-565960b20a9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.364207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed3b75-04e3-4309-bac3-565960b20a9d service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.364482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed3b75-04e3-4309-bac3-565960b20a9d service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.365067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed3b75-04e3-4309-bac3-565960b20a9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.375589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed3b75-04e3-4309-bac3-565960b20a9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.377433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed3b75-04e3-4309-bac3-565960b20a9d service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.389862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 933/1846] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:47.392059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0896b-9564-46cd-9717-35ce56a5a5a6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.392059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0896b-9564-46cd-9717-35ce56a5a5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.392059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0896b-9564-46cd-9717-35ce56a5a5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.422476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.423230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.423546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.424144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] PATH_INFO: `/v3/users/2db27aea5c6640e4b6235897589c14d6` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.425692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db27aea5c6640e4b6235897589c14d6)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.426438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-35c0896b-9564-46cd-9717-35ce56a5a5a6 None None] MFA Rules not processed for user `457ea559dd084aec86d2c46e0baba76d`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:47.427625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.427928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-35c0896b-9564-46cd-9717-35ce56a5a5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.502655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db27aea5c6640e4b6235897589c14d6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:47.513538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db27aea5c6640e4b6235897589c14d6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:47.514669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db27aea5c6640e4b6235897589c14d6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:47.515291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db27aea5c6640e4b6235897589c14d6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:47.535829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db27aea5c6640e4b6235897589c14d6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:47.540786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 934/1847] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:47 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:47.540786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3b721-c269-45ef-9480-fa0227f805ec None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.540786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3b721-c269-45ef-9480-fa0227f805ec None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.540786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3b721-c269-45ef-9480-fa0227f805ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.563087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 914/1848] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:47 2022] DELETE /identity/v3/users/2db27aea5c6640e4b6235897589c14d6 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:47.564239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.566281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.582182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-80a3b721-c269-45ef-9480-fa0227f805ec None None] MFA Rules not processed for user `76961abde5d7401488285ac649ff5b61`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:47.583705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-80a3b721-c269-45ef-9480-fa0227f805ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.650405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085903c-c2e0-4bd0-a1f7-39473f01af17 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.655712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951c022-836b-4641-bf1e-6da238438e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.657027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951c022-836b-4641-bf1e-6da238438e45 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.657433 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951c022-836b-4641-bf1e-6da238438e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.657816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951c022-836b-4641-bf1e-6da238438e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.658556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951c022-836b-4641-bf1e-6da238438e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.661610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951c022-836b-4641-bf1e-6da238438e45 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.667189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:47.714255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 935/1849] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:47.716066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728f86c-2e9b-4c89-86c3-f5638df424df None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.716487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728f86c-2e9b-4c89-86c3-f5638df424df None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.716886 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728f86c-2e9b-4c89-86c3-f5638df424df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.718631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 915/1850] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:47 2022] POST /identity/v3/projects => generated 407 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:47.722510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951c022-836b-4641-bf1e-6da238438e45 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.726960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e951c022-836b-4641-bf1e-6da238438e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.758960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0728f86c-2e9b-4c89-86c3-f5638df424df None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:47.763347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0728f86c-2e9b-4c89-86c3-f5638df424df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.837311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:47.865299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951c022-836b-4641-bf1e-6da238438e45 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.872975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d109d2-9b6d-4035-b429-5b747dd4a7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.873924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d109d2-9b6d-4035-b429-5b747dd4a7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.874460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d109d2-9b6d-4035-b429-5b747dd4a7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.874878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d109d2-9b6d-4035-b429-5b747dd4a7c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.875798 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d109d2-9b6d-4035-b429-5b747dd4a7c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.878302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d109d2-9b6d-4035-b429-5b747dd4a7c8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.881781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 936/1851] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:47.882608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728f86c-2e9b-4c89-86c3-f5638df424df None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.900924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728f86c-2e9b-4c89-86c3-f5638df424df None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.914539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81877e95-c37d-4052-bfa8-6891ecafd6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.915195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81877e95-c37d-4052-bfa8-6891ecafd6ea admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.915473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81877e95-c37d-4052-bfa8-6891ecafd6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.915736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81877e95-c37d-4052-bfa8-6891ecafd6ea admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.916373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81877e95-c37d-4052-bfa8-6891ecafd6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.919013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81877e95-c37d-4052-bfa8-6891ecafd6ea admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.943868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 916/1852] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:47 2022] POST /identity/v3/projects => generated 365 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:47.944865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d109d2-9b6d-4035-b429-5b747dd4a7c8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.946907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-09d109d2-9b6d-4035-b429-5b747dd4a7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.970289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 937/1853] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:47 2022] POST /identity/v3/users => generated 310 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:47.970289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81877e95-c37d-4052-bfa8-6891ecafd6ea admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.980867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81877e95-c37d-4052-bfa8-6891ecafd6ea admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.019866 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d109d2-9b6d-4035-b429-5b747dd4a7c8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.024317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb855c6d-ee1c-4f6f-976d-c32c11135038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.025613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb855c6d-ee1c-4f6f-976d-c32c11135038 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.025890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb855c6d-ee1c-4f6f-976d-c32c11135038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.026165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb855c6d-ee1c-4f6f-976d-c32c11135038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.066474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c449dd-957c-44f2-8646-df03d4d0d7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.067252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c449dd-957c-44f2-8646-df03d4d0d7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.067638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c449dd-957c-44f2-8646-df03d4d0d7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.068031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c449dd-957c-44f2-8646-df03d4d0d7c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.071069 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c449dd-957c-44f2-8646-df03d4d0d7c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.075360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c449dd-957c-44f2-8646-df03d4d0d7c7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.126091 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb855c6d-ee1c-4f6f-976d-c32c11135038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.127679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb855c6d-ee1c-4f6f-976d-c32c11135038 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.130463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 938/1854] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:47 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:48.131608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c449dd-957c-44f2-8646-df03d4d0d7c7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.147338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c449dd-957c-44f2-8646-df03d4d0d7c7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.150314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 917/1855] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:48.152860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb855c6d-ee1c-4f6f-976d-c32c11135038 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.153959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787a00-9d9b-4895-a3ee-ee9c69ebffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.154976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787a00-9d9b-4895-a3ee-ee9c69ebffec admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.155449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787a00-9d9b-4895-a3ee-ee9c69ebffec admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.155842 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787a00-9d9b-4895-a3ee-ee9c69ebffec admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.156832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c787a00-9d9b-4895-a3ee-ee9c69ebffec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.159447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c787a00-9d9b-4895-a3ee-ee9c69ebffec admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.175468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb855c6d-ee1c-4f6f-976d-c32c11135038 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.181200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 939/1856] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:48.182255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787a00-9d9b-4895-a3ee-ee9c69ebffec admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.184311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70a2d9-ccc0-49cc-b427-f1ac4be7d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.185913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70a2d9-ccc0-49cc-b427-f1ac4be7d2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.186314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70a2d9-ccc0-49cc-b427-f1ac4be7d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.186682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70a2d9-ccc0-49cc-b427-f1ac4be7d2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.187626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70a2d9-ccc0-49cc-b427-f1ac4be7d2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.190250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70a2d9-ccc0-49cc-b427-f1ac4be7d2d4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.195812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787a00-9d9b-4895-a3ee-ee9c69ebffec admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.201264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206432e0-bbff-481d-8e03-d91862fcb957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.202294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206432e0-bbff-481d-8e03-d91862fcb957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.217839 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206432e0-bbff-481d-8e03-d91862fcb957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.219376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206432e0-bbff-481d-8e03-d91862fcb957 admin admin] PATH_INFO: `/v3/projects/c560627674ee4116ad7071cfab067859/users/a26b41d5755c467caaa9a26cdd7340b2/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.224050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 918/1857] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:48.227079 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70a2d9-ccc0-49cc-b427-f1ac4be7d2d4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.273679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206432e0-bbff-481d-8e03-d91862fcb957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c560627674ee4116ad7071cfab067859, user_id=a26b41d5755c467caaa9a26cdd7340b2, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.279338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206432e0-bbff-481d-8e03-d91862fcb957 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.291101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70a2d9-ccc0-49cc-b427-f1ac4be7d2d4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.302104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:48.303826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 940/1858] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:48 2022] PUT /identity/v3/projects/c560627674ee4116ad7071cfab067859/users/a26b41d5755c467caaa9a26cdd7340b2/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:48.308256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206432e0-bbff-481d-8e03-d91862fcb957 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.404263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926001e-f6b5-4785-83e0-4341258067f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.407061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926001e-f6b5-4785-83e0-4341258067f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.407643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926001e-f6b5-4785-83e0-4341258067f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.407948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926001e-f6b5-4785-83e0-4341258067f9 admin admin] PATH_INFO: `/v3/projects/e64abef6418b4f6c86e0ee09cbc39a7e/users/c1b07978810a451e9e1c9309f487fb61/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.429943 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206432e0-bbff-481d-8e03-d91862fcb957 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.435991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50986332-0291-41c9-a4b0-adcdb4e13271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.436916 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50986332-0291-41c9-a4b0-adcdb4e13271 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.438223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50986332-0291-41c9-a4b0-adcdb4e13271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.438490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50986332-0291-41c9-a4b0-adcdb4e13271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.439279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50986332-0291-41c9-a4b0-adcdb4e13271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.441525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50986332-0291-41c9-a4b0-adcdb4e13271 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.447478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926001e-f6b5-4785-83e0-4341258067f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64abef6418b4f6c86e0ee09cbc39a7e, user_id=c1b07978810a451e9e1c9309f487fb61, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.454644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 941/1859] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:48.459382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926001e-f6b5-4785-83e0-4341258067f9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.460779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50986332-0291-41c9-a4b0-adcdb4e13271 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.487774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 919/1860] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:48 2022] PUT /identity/v3/projects/e64abef6418b4f6c86e0ee09cbc39a7e/users/c1b07978810a451e9e1c9309f487fb61/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:48.494138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926001e-f6b5-4785-83e0-4341258067f9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.495925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50986332-0291-41c9-a4b0-adcdb4e13271 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.527365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:48.617382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0eb7e-e867-46f9-b6a9-6986dcbf8130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.618173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0eb7e-e867-46f9-b6a9-6986dcbf8130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.618686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0eb7e-e867-46f9-b6a9-6986dcbf8130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.619060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0eb7e-e867-46f9-b6a9-6986dcbf8130 admin admin] PATH_INFO: `/v3/projects/c560627674ee4116ad7071cfab067859/users/a26b41d5755c467caaa9a26cdd7340b2/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.621707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0eb7e-e867-46f9-b6a9-6986dcbf8130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c560627674ee4116ad7071cfab067859, user_id=a26b41d5755c467caaa9a26cdd7340b2, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.624646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0eb7e-e867-46f9-b6a9-6986dcbf8130 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.634982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926001e-f6b5-4785-83e0-4341258067f9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.638444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 942/1861] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:48 2022] PUT /identity/v3/projects/c560627674ee4116ad7071cfab067859/users/a26b41d5755c467caaa9a26cdd7340b2/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:48.652272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022729e-334a-426f-9ac6-50bbdb6b9af9 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.652272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022729e-334a-426f-9ac6-50bbdb6b9af9 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.653042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022729e-334a-426f-9ac6-50bbdb6b9af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.717129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3022729e-334a-426f-9ac6-50bbdb6b9af9 None None] MFA Rules not processed for user `a26b41d5755c467caaa9a26cdd7340b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:48.718384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3022729e-334a-426f-9ac6-50bbdb6b9af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:48.736378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e1443-50e8-4f00-b0ae-d6f0c130585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.737325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e1443-50e8-4f00-b0ae-d6f0c130585d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.737833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e1443-50e8-4f00-b0ae-d6f0c130585d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.738211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e1443-50e8-4f00-b0ae-d6f0c130585d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.738970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e1443-50e8-4f00-b0ae-d6f0c130585d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.741618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e1443-50e8-4f00-b0ae-d6f0c130585d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.758517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 920/1862] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:48.759541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e1443-50e8-4f00-b0ae-d6f0c130585d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.778685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e1443-50e8-4f00-b0ae-d6f0c130585d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.787609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53746118-3674-417d-ab3c-bc789554bede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.788977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53746118-3674-417d-ab3c-bc789554bede admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.789309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53746118-3674-417d-ab3c-bc789554bede admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.789586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53746118-3674-417d-ab3c-bc789554bede admin admin] PATH_INFO: `/v3/projects/e64abef6418b4f6c86e0ee09cbc39a7e/users/c1b07978810a451e9e1c9309f487fb61/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.793094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53746118-3674-417d-ab3c-bc789554bede admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64abef6418b4f6c86e0ee09cbc39a7e, user_id=c1b07978810a451e9e1c9309f487fb61, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.796513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53746118-3674-417d-ab3c-bc789554bede admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.811338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 921/1863] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:48 2022] PUT /identity/v3/projects/e64abef6418b4f6c86e0ee09cbc39a7e/users/c1b07978810a451e9e1c9309f487fb61/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:48.812147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53746118-3674-417d-ab3c-bc789554bede admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.814721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-53746118-3674-417d-ab3c-bc789554bede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:48.833131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 943/1864] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:48 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:48.834996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c8ae9-6b35-4d5a-882c-3e18bcb8ed6e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.835284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c8ae9-6b35-4d5a-882c-3e18bcb8ed6e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.835970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c8ae9-6b35-4d5a-882c-3e18bcb8ed6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.851160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:48.869972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-189c8ae9-6b35-4d5a-882c-3e18bcb8ed6e None None] MFA Rules not processed for user `c1b07978810a451e9e1c9309f487fb61`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:48.871210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-189c8ae9-6b35-4d5a-882c-3e18bcb8ed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:48.898905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53746118-3674-417d-ab3c-bc789554bede admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:48.911874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:48.912513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:48.912787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:48.913053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] PATH_INFO: `/v3/projects/297933af06fe49d99834d31c4aeb193e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:48.914235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=297933af06fe49d99834d31c4aeb193e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:48.917111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:48.957314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 944/1865] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:48.958972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c8ae9-6b35-4d5a-882c-3e18bcb8ed6e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:48.961352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-189c8ae9-6b35-4d5a-882c-3e18bcb8ed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:48.977384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297933af06fe49d99834d31c4aeb193e', 'request_id': 'req-000016cf-38b9-4b7b-833a-515fa3f3e4ea'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:48.988258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:48.988631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297933af06fe49d99834d31c4aeb193e', 'request_id': 'req-000016cf-38b9-4b7b-833a-515fa3f3e4ea'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.002030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297933af06fe49d99834d31c4aeb193e', 'request_id': 'req-000016cf-38b9-4b7b-833a-515fa3f3e4ea'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.019368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] The token cache is being invalidate because project 297933af06fe49d99834d31c4aeb193e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:49.019659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.020332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.021882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 922/1866] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:48 2022] DELETE /identity/v3/projects/297933af06fe49d99834d31c4aeb193e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:49.024681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:49.024681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:49.028071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c8ae9-6b35-4d5a-882c-3e18bcb8ed6e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:49.081275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:49.081908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.082188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.082334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000016cf-38b9-4b7b-833a-515fa3f3e4ea admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:49.082456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.084275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:49.086116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9896c3f-b779-4494-921c-a06f02f90895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:49.086834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9896c3f-b779-4494-921c-a06f02f90895 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.087204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9896c3f-b779-4494-921c-a06f02f90895 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.087496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9896c3f-b779-4494-921c-a06f02f90895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.089223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d9896c3f-b779-4494-921c-a06f02f90895 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:49.150735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:49.152181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:49.157926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9896c3f-b779-4494-921c-a06f02f90895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:49.159650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9896c3f-b779-4494-921c-a06f02f90895 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:49.164358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 945/1867] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:48 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:49.165290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:49.166622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:49.170290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 923/1868] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:49.260737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa5433-21d5-4fd2-a9af-f3723a29d5ee service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:49.267575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:49.268318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.268695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.269064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] PATH_INFO: `/v3/users/42b76617c18649b1afb0d85fc1df11ad` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.270259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b76617c18649b1afb0d85fc1df11ad)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:49.272226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:49.278903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9896c3f-b779-4494-921c-a06f02f90895 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:49.284314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:49.289747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9896c3f-b779-4494-921c-a06f02f90895 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:49.301891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499063f0-4d97-490f-901b-1733d8d43a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:49.302434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499063f0-4d97-490f-901b-1733d8d43a2a service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.302711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499063f0-4d97-490f-901b-1733d8d43a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.302976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499063f0-4d97-490f-901b-1733d8d43a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.305417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-499063f0-4d97-490f-901b-1733d8d43a2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:49.313495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:49.337497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b76617c18649b1afb0d85fc1df11ad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.347645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b76617c18649b1afb0d85fc1df11ad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.356860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b76617c18649b1afb0d85fc1df11ad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.357884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b76617c18649b1afb0d85fc1df11ad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.373975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b76617c18649b1afb0d85fc1df11ad'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.381431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 946/1869] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:49 2022] DELETE /identity/v3/users/42b76617c18649b1afb0d85fc1df11ad => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:49.411344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:49.412786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:49.428309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499063f0-4d97-490f-901b-1733d8d43a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:49.429793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499063f0-4d97-490f-901b-1733d8d43a2a service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:49.442995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 924/1870] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:01:49.487786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84fabb-65e0-4412-b980-d5fb9a603eef admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:49.494277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d783f27-b3ce-4b54-ad77-f62a6451aed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:49.495152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d783f27-b3ce-4b54-ad77-f62a6451aed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.495437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d783f27-b3ce-4b54-ad77-f62a6451aed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.495706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d783f27-b3ce-4b54-ad77-f62a6451aed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.498780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5d783f27-b3ce-4b54-ad77-f62a6451aed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:49.584956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d783f27-b3ce-4b54-ad77-f62a6451aed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:49.588172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d783f27-b3ce-4b54-ad77-f62a6451aed1 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:49.608992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 947/1871] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:49 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:01:49.737092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499063f0-4d97-490f-901b-1733d8d43a2a service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:49.752277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499063f0-4d97-490f-901b-1733d8d43a2a service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:49.824774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:49.825459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.825740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.826017 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] PATH_INFO: `/v3/projects/5f21a70ce0e14ca8bff3e3109d00e70e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.827297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f21a70ce0e14ca8bff3e3109d00e70e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:49.829577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:49.885171 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f21a70ce0e14ca8bff3e3109d00e70e', 'request_id': 'req-304cc8c1-7edd-4356-82e4-5821135972a7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.892171 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f21a70ce0e14ca8bff3e3109d00e70e', 'request_id': 'req-304cc8c1-7edd-4356-82e4-5821135972a7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.902215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f21a70ce0e14ca8bff3e3109d00e70e', 'request_id': 'req-304cc8c1-7edd-4356-82e4-5821135972a7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.916399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] The token cache is being invalidate because project 5f21a70ce0e14ca8bff3e3109d00e70e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:49.916707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.917548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-304cc8c1-7edd-4356-82e4-5821135972a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:49.923989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 925/1872] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:49 2022] DELETE /identity/v3/projects/5f21a70ce0e14ca8bff3e3109d00e70e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:49.926811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03231f-efa6-4ee5-b6be-f7c564f29c05 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.927561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03231f-efa6-4ee5-b6be-f7c564f29c05 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.928372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03231f-efa6-4ee5-b6be-f7c564f29c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.954071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-eb03231f-efa6-4ee5-b6be-f7c564f29c05 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:49.955467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-eb03231f-efa6-4ee5-b6be-f7c564f29c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:50.021285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.034515 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 948/1873] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:50.041152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8ddd4-b9e7-4987-a69e-75d05f0bae08 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.041152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8ddd4-b9e7-4987-a69e-75d05f0bae08 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.041590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8ddd4-b9e7-4987-a69e-75d05f0bae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.064476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7cf8ddd4-b9e7-4987-a69e-75d05f0bae08 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:50.065942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7cf8ddd4-b9e7-4987-a69e-75d05f0bae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:50.073977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 926/1874] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:50.078860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03231f-efa6-4ee5-b6be-f7c564f29c05 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.088946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03231f-efa6-4ee5-b6be-f7c564f29c05 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.093825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49713064-efd0-46df-a0be-bc16e2cdc0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.094667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713064-efd0-46df-a0be-bc16e2cdc0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.094950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713064-efd0-46df-a0be-bc16e2cdc0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.095250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713064-efd0-46df-a0be-bc16e2cdc0f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.095929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49713064-efd0-46df-a0be-bc16e2cdc0f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:50.096235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49713064-efd0-46df-a0be-bc16e2cdc0f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.097932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49713064-efd0-46df-a0be-bc16e2cdc0f7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.112104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 949/1875] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:50.116178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8ddd4-b9e7-4987-a69e-75d05f0bae08 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.126295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8ddd4-b9e7-4987-a69e-75d05f0bae08 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.131204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096435d-1405-4be2-a0a6-e3137a400b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.131816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096435d-1405-4be2-a0a6-e3137a400b7a admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.132098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096435d-1405-4be2-a0a6-e3137a400b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.132371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096435d-1405-4be2-a0a6-e3137a400b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.133022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096435d-1405-4be2-a0a6-e3137a400b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.137682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096435d-1405-4be2-a0a6-e3137a400b7a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.168815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 927/1876] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:50 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:50.173977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49713064-efd0-46df-a0be-bc16e2cdc0f7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.192280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49713064-efd0-46df-a0be-bc16e2cdc0f7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.249733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f2807-cb98-4eeb-8220-3f1fc12ef2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.250857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f2807-cb98-4eeb-8220-3f1fc12ef2bf admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.251203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f2807-cb98-4eeb-8220-3f1fc12ef2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.251479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f2807-cb98-4eeb-8220-3f1fc12ef2bf admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.252137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f2807-cb98-4eeb-8220-3f1fc12ef2bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.254094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f2807-cb98-4eeb-8220-3f1fc12ef2bf admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.276124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 950/1877] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:50 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:50.280030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096435d-1405-4be2-a0a6-e3137a400b7a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.285742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.292239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096435d-1405-4be2-a0a6-e3137a400b7a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.298650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0191f-8bf3-4bc5-957e-b54f3818d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.299249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0191f-8bf3-4bc5-957e-b54f3818d75f admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.299530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0191f-8bf3-4bc5-957e-b54f3818d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.299802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0191f-8bf3-4bc5-957e-b54f3818d75f admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.300443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0191f-8bf3-4bc5-957e-b54f3818d75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.302051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0191f-8bf3-4bc5-957e-b54f3818d75f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.311921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 928/1878] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:50.316426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f2807-cb98-4eeb-8220-3f1fc12ef2bf admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.326174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f2807-cb98-4eeb-8220-3f1fc12ef2bf admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.331148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a36d71-730a-4797-ad7a-130a77b7f8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.331738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a36d71-730a-4797-ad7a-130a77b7f8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.332015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a36d71-730a-4797-ad7a-130a77b7f8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.332282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a36d71-730a-4797-ad7a-130a77b7f8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.332917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a36d71-730a-4797-ad7a-130a77b7f8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.334530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a36d71-730a-4797-ad7a-130a77b7f8e6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.345377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 951/1879] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:50.349848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0191f-8bf3-4bc5-957e-b54f3818d75f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.360585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0191f-8bf3-4bc5-957e-b54f3818d75f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.365227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69822c27-ca68-484d-ad6d-830d47da84a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.365869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69822c27-ca68-484d-ad6d-830d47da84a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.366152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69822c27-ca68-484d-ad6d-830d47da84a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.366429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69822c27-ca68-484d-ad6d-830d47da84a6 admin admin] PATH_INFO: `/v3/projects/f75824b89f8e479d8ab387df96644b73/users/d3d58b05c8ff4fa0a3b5f33dae578170/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.381313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69822c27-ca68-484d-ad6d-830d47da84a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75824b89f8e479d8ab387df96644b73, user_id=d3d58b05c8ff4fa0a3b5f33dae578170, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.383971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69822c27-ca68-484d-ad6d-830d47da84a6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.400381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 929/1880] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:50 2022] PUT /identity/v3/projects/f75824b89f8e479d8ab387df96644b73/users/d3d58b05c8ff4fa0a3b5f33dae578170/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:50.411030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ad6f4-bd38-4f52-9c62-31e196830895 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.411358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ad6f4-bd38-4f52-9c62-31e196830895 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.411649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ad6f4-bd38-4f52-9c62-31e196830895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.442841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cc2ad6f4-bd38-4f52-9c62-31e196830895 None None] MFA Rules not processed for user `d3d58b05c8ff4fa0a3b5f33dae578170`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:50.443977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cc2ad6f4-bd38-4f52-9c62-31e196830895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:50.522474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 952/1881] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:50.534184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69822c27-ca68-484d-ad6d-830d47da84a6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.536052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-69822c27-ca68-484d-ad6d-830d47da84a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:50.607972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69822c27-ca68-484d-ad6d-830d47da84a6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.613456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3277f4-12ff-4a8b-829d-90ce0047f6d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.614126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3277f4-12ff-4a8b-829d-90ce0047f6d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.614408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3277f4-12ff-4a8b-829d-90ce0047f6d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.614675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3277f4-12ff-4a8b-829d-90ce0047f6d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.630018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3277f4-12ff-4a8b-829d-90ce0047f6d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.631661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3277f4-12ff-4a8b-829d-90ce0047f6d4 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.643588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 930/1882] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:50.671580 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ad6f4-bd38-4f52-9c62-31e196830895 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.743889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.749577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ad6f4-bd38-4f52-9c62-31e196830895 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.756625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.757361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.757927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.758201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] PATH_INFO: `/v3/users/d3d58b05c8ff4fa0a3b5f33dae578170` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.759394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3d58b05c8ff4fa0a3b5f33dae578170)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.761706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.810348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3d58b05c8ff4fa0a3b5f33dae578170'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:50.819913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3d58b05c8ff4fa0a3b5f33dae578170'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:50.827882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3d58b05c8ff4fa0a3b5f33dae578170'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:50.828581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3d58b05c8ff4fa0a3b5f33dae578170'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:50.843022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3d58b05c8ff4fa0a3b5f33dae578170'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:50.850774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 953/1883] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:50 2022] DELETE /identity/v3/users/d3d58b05c8ff4fa0a3b5f33dae578170 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:50.855490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f3c1e-dcc4-483f-925f-7d3dc3594d92 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.855991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f3c1e-dcc4-483f-925f-7d3dc3594d92 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.856332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f3c1e-dcc4-483f-925f-7d3dc3594d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.876817 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-343f3c1e-dcc4-483f-925f-7d3dc3594d92 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:50.878065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-343f3c1e-dcc4-483f-925f-7d3dc3594d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:50.944160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 931/1884] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:50.953999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.955434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:51.019504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e231e-30ec-4faa-a58b-36f35124a563 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:51.023200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4d6db-b25f-4a82-8d06-37da53e0b5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:51.023807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d6db-b25f-4a82-8d06-37da53e0b5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:51.024082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d6db-b25f-4a82-8d06-37da53e0b5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:51.024386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d6db-b25f-4a82-8d06-37da53e0b5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:51.036871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4d6db-b25f-4a82-8d06-37da53e0b5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:51.038349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4d6db-b25f-4a82-8d06-37da53e0b5c5 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:51.048758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 954/1885] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:51.486849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f3c1e-dcc4-483f-925f-7d3dc3594d92 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:51.495431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-343f3c1e-dcc4-483f-925f-7d3dc3594d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:51.509413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f3c1e-dcc4-483f-925f-7d3dc3594d92 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:51.514746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:51.515957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:51.516248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:51.516517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] PATH_INFO: `/v3/projects/f75824b89f8e479d8ab387df96644b73` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:51.517712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f75824b89f8e479d8ab387df96644b73)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:51.519693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:51.573394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f75824b89f8e479d8ab387df96644b73', 'request_id': 'req-596b2396-5269-4a20-8ad3-6e7660c50570'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:51.581129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f75824b89f8e479d8ab387df96644b73', 'request_id': 'req-596b2396-5269-4a20-8ad3-6e7660c50570'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:51.589638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f75824b89f8e479d8ab387df96644b73', 'request_id': 'req-596b2396-5269-4a20-8ad3-6e7660c50570'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:51.594372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:51.602779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] The token cache is being invalidate because project f75824b89f8e479d8ab387df96644b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:51.603081 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:51.604191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-596b2396-5269-4a20-8ad3-6e7660c50570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:51.606170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 932/1886] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:51 2022] DELETE /identity/v3/projects/f75824b89f8e479d8ab387df96644b73 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:51.612765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9ba65-1073-4330-bacc-bd9dfd94a3e5 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:51.613299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9ba65-1073-4330-bacc-bd9dfd94a3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:51.613573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9ba65-1073-4330-bacc-bd9dfd94a3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:51.636753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7ae9ba65-1073-4330-bacc-bd9dfd94a3e5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:51.638593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7ae9ba65-1073-4330-bacc-bd9dfd94a3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:51.806823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 955/1887] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:51.809288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd88475-420f-4093-9d62-21dbc4b70dc8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:51.809583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd88475-420f-4093-9d62-21dbc4b70dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:51.810239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd88475-420f-4093-9d62-21dbc4b70dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:51.851596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6dd88475-420f-4093-9d62-21dbc4b70dc8 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:51.854520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6dd88475-420f-4093-9d62-21dbc4b70dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:51.877898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 933/1888] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:51.882006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9ba65-1073-4330-bacc-bd9dfd94a3e5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:51.898650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9ba65-1073-4330-bacc-bd9dfd94a3e5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:51.923080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55331e90-7ef2-4a73-a292-3a868ab1a3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:51.923773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55331e90-7ef2-4a73-a292-3a868ab1a3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:51.924066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55331e90-7ef2-4a73-a292-3a868ab1a3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:51.924335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55331e90-7ef2-4a73-a292-3a868ab1a3f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:51.924992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55331e90-7ef2-4a73-a292-3a868ab1a3f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:51.930415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55331e90-7ef2-4a73-a292-3a868ab1a3f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:51.934568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55331e90-7ef2-4a73-a292-3a868ab1a3f4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:51.983540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 956/1889] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:51.985127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd88475-420f-4093-9d62-21dbc4b70dc8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:51.998506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd88475-420f-4093-9d62-21dbc4b70dc8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:52.012155 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d97ac-a7a3-4bfb-9c24-81e208c0b2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:52.012474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d97ac-a7a3-4bfb-9c24-81e208c0b2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.012936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d97ac-a7a3-4bfb-9c24-81e208c0b2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.013245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d97ac-a7a3-4bfb-9c24-81e208c0b2d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.013895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d97ac-a7a3-4bfb-9c24-81e208c0b2d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:52.016451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d97ac-a7a3-4bfb-9c24-81e208c0b2d7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:52.120933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 934/1890] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:51 2022] POST /identity/v3/projects => generated 361 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:52.130390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55331e90-7ef2-4a73-a292-3a868ab1a3f4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:52.138675 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:52.152280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55331e90-7ef2-4a73-a292-3a868ab1a3f4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:52.233368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3326-b692-44e4-919f-1f26590568a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:52.234121 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3326-b692-44e4-919f-1f26590568a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.234693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3326-b692-44e4-919f-1f26590568a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.235105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3326-b692-44e4-919f-1f26590568a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.235783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa3326-b692-44e4-919f-1f26590568a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:52.237774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa3326-b692-44e4-919f-1f26590568a0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:52.272452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 957/1891] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:52 2022] POST /identity/v3/users => generated 281 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:52.276474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d97ac-a7a3-4bfb-9c24-81e208c0b2d7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:52.295377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d97ac-a7a3-4bfb-9c24-81e208c0b2d7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:52.303708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff6c6d-695e-48dc-9486-0c6eb09f62d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:52.304915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff6c6d-695e-48dc-9486-0c6eb09f62d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.305255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff6c6d-695e-48dc-9486-0c6eb09f62d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.305494 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff6c6d-695e-48dc-9486-0c6eb09f62d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.306203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff6c6d-695e-48dc-9486-0c6eb09f62d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:52.308362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff6c6d-695e-48dc-9486-0c6eb09f62d3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:52.325689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 935/1892] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:52.330901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3326-b692-44e4-919f-1f26590568a0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:52.344969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3326-b692-44e4-919f-1f26590568a0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:52.352877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc981f1-c54c-4e27-aa22-fa1089552d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:52.353409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc981f1-c54c-4e27-aa22-fa1089552d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.353710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc981f1-c54c-4e27-aa22-fa1089552d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.353962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc981f1-c54c-4e27-aa22-fa1089552d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.354718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc981f1-c54c-4e27-aa22-fa1089552d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:52.356317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc981f1-c54c-4e27-aa22-fa1089552d58 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:52.370310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 958/1893] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:01:52.375458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff6c6d-695e-48dc-9486-0c6eb09f62d3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:52.386430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff6c6d-695e-48dc-9486-0c6eb09f62d3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:52.391981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa2724-cf64-440e-a77d-0228705fcb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:52.392567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa2724-cf64-440e-a77d-0228705fcb5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.392853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa2724-cf64-440e-a77d-0228705fcb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.393167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa2724-cf64-440e-a77d-0228705fcb5e admin admin] PATH_INFO: `/v3/projects/0c6403e5ecc44c52ba972df67b70037e/users/f6c3c0c190b842feaad823008273c085/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.414943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa2724-cf64-440e-a77d-0228705fcb5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6403e5ecc44c52ba972df67b70037e, user_id=f6c3c0c190b842feaad823008273c085, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:52.417892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa2724-cf64-440e-a77d-0228705fcb5e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:52.441543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 936/1894] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:52 2022] PUT /identity/v3/projects/0c6403e5ecc44c52ba972df67b70037e/users/f6c3c0c190b842feaad823008273c085/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:52.446992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7df67c-1d97-44f2-a447-08fae5370247 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.447285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7df67c-1d97-44f2-a447-08fae5370247 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.447563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7df67c-1d97-44f2-a447-08fae5370247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.477665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6e7df67c-1d97-44f2-a447-08fae5370247 None None] MFA Rules not processed for user `f6c3c0c190b842feaad823008273c085`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:52.478889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6e7df67c-1d97-44f2-a447-08fae5370247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:52.559408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 959/1895] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:52.596942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa2724-cf64-440e-a77d-0228705fcb5e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:52.598740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0aaa2724-cf64-440e-a77d-0228705fcb5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:52.676894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa2724-cf64-440e-a77d-0228705fcb5e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:52.680694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939009d-e761-480b-8599-194cafbd6094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:52.681335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939009d-e761-480b-8599-194cafbd6094 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.681616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939009d-e761-480b-8599-194cafbd6094 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.681888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939009d-e761-480b-8599-194cafbd6094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.691172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939009d-e761-480b-8599-194cafbd6094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:52.692596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939009d-e761-480b-8599-194cafbd6094 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:52.701828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 937/1896] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:01:52.716850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7df67c-1d97-44f2-a447-08fae5370247 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:52.780128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7df67c-1d97-44f2-a447-08fae5370247 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:52.785539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:52.786161 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.786435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.786703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] PATH_INFO: `/v3/users/f6c3c0c190b842feaad823008273c085` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.787828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c3c0c190b842feaad823008273c085)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:52.789777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:52.803864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:52.836397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c3c0c190b842feaad823008273c085'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:52.844367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c3c0c190b842feaad823008273c085'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:52.851590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c3c0c190b842feaad823008273c085'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:52.852214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c3c0c190b842feaad823008273c085'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:52.865685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c3c0c190b842feaad823008273c085'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:52.871411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 960/1897] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:52 2022] DELETE /identity/v3/users/f6c3c0c190b842feaad823008273c085 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:52.876101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf0c9-9dfb-411c-a3b9-a021950de6bf None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.876414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf0c9-9dfb-411c-a3b9-a021950de6bf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.876687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf0c9-9dfb-411c-a3b9-a021950de6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.895308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b55bf0c9-9dfb-411c-a3b9-a021950de6bf None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:52.896498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b55bf0c9-9dfb-411c-a3b9-a021950de6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:52.954607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 938/1898] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:52.964719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:52.966152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.024060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403569b-3046-4962-a4cb-ff5297e5764d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:53.027620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb094f3-032f-442a-8567-d96d33718d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:53.028224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb094f3-032f-442a-8567-d96d33718d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.028504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb094f3-032f-442a-8567-d96d33718d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.028773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb094f3-032f-442a-8567-d96d33718d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.038915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb094f3-032f-442a-8567-d96d33718d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:53.040352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb094f3-032f-442a-8567-d96d33718d55 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:53.052205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 961/1899] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:53.130078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf0c9-9dfb-411c-a3b9-a021950de6bf None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:53.132191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b55bf0c9-9dfb-411c-a3b9-a021950de6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.144927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf0c9-9dfb-411c-a3b9-a021950de6bf None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:53.153204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:53.154101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.154479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.154851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] PATH_INFO: `/v3/users/76961abde5d7401488285ac649ff5b61` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.156981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] RBAC: Authorizing `identity:delete_user(user_id=76961abde5d7401488285ac649ff5b61)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:53.160069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:53.193983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:53.215739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76961abde5d7401488285ac649ff5b61'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.224322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76961abde5d7401488285ac649ff5b61'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.224960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76961abde5d7401488285ac649ff5b61'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.225612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76961abde5d7401488285ac649ff5b61'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.238534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76961abde5d7401488285ac649ff5b61'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.264431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 939/1900] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:53 2022] DELETE /identity/v3/users/76961abde5d7401488285ac649ff5b61 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:53.269722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58182a93-923d-46c7-b199-b70949d59e32 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.270060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58182a93-923d-46c7-b199-b70949d59e32 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.270342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58182a93-923d-46c7-b199-b70949d59e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.299600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-58182a93-923d-46c7-b199-b70949d59e32 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:53.300819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-58182a93-923d-46c7-b199-b70949d59e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.375273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 962/1901] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:53.399852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:53.401571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.461152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58182a93-923d-46c7-b199-b70949d59e32 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:53.461152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-58182a93-923d-46c7-b199-b70949d59e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.472737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58182a93-923d-46c7-b199-b70949d59e32 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:53.485714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:53.487217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.488274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.488788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] PATH_INFO: `/v3/projects/0c6403e5ecc44c52ba972df67b70037e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.490500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6403e5ecc44c52ba972df67b70037e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:53.502659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:53.517493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567d1d1-34eb-4bae-afe0-a81d4f27a07c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:53.521499 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7882f82-e120-4a12-985d-6faefa063b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:53.522246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7882f82-e120-4a12-985d-6faefa063b7d service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.522634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7882f82-e120-4a12-985d-6faefa063b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.522871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7882f82-e120-4a12-985d-6faefa063b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.539716 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7882f82-e120-4a12-985d-6faefa063b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:53.541186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7882f82-e120-4a12-985d-6faefa063b7d service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:53.555173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 940/1902] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:53.561389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:53.582411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6403e5ecc44c52ba972df67b70037e', 'request_id': 'req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.593236 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6403e5ecc44c52ba972df67b70037e', 'request_id': 'req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.604584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6403e5ecc44c52ba972df67b70037e', 'request_id': 'req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.623102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] The token cache is being invalidate because project 0c6403e5ecc44c52ba972df67b70037e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:53.623488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.624229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-eb2f11ed-17c5-4e39-a708-ca7a52cdfd5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:53.626168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 963/1903] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:53 2022] DELETE /identity/v3/projects/0c6403e5ecc44c52ba972df67b70037e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:53.636051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171bf6-0faa-413a-8c6a-ff8452ea4554 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.636561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171bf6-0faa-413a-8c6a-ff8452ea4554 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.636662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171bf6-0faa-413a-8c6a-ff8452ea4554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.662542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6e171bf6-0faa-413a-8c6a-ff8452ea4554 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:53.663928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6e171bf6-0faa-413a-8c6a-ff8452ea4554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.749894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 941/1904] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:53.751282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a770-6c89-46f9-9f7d-805be2636320 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.751568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a770-6c89-46f9-9f7d-805be2636320 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.751841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a770-6c89-46f9-9f7d-805be2636320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.784342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5b31a770-6c89-46f9-9f7d-805be2636320 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:53.785880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5b31a770-6c89-46f9-9f7d-805be2636320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.798559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 964/1905] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:53.802731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e171bf6-0faa-413a-8c6a-ff8452ea4554 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:53.814406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e171bf6-0faa-413a-8c6a-ff8452ea4554 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:53.819466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce02057-8602-4eda-bbc5-24ece3a29bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:53.820065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce02057-8602-4eda-bbc5-24ece3a29bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.820339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce02057-8602-4eda-bbc5-24ece3a29bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.820604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce02057-8602-4eda-bbc5-24ece3a29bc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.821274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce02057-8602-4eda-bbc5-24ece3a29bc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:53.821611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce02057-8602-4eda-bbc5-24ece3a29bc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:53.824140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce02057-8602-4eda-bbc5-24ece3a29bc6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:53.837885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 942/1906] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:53.841989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31a770-6c89-46f9-9f7d-805be2636320 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:53.851805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31a770-6c89-46f9-9f7d-805be2636320 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:53.856973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d46df-e980-445d-908f-29867ff10022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:53.857696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d46df-e980-445d-908f-29867ff10022 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.858363 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d46df-e980-445d-908f-29867ff10022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.858363 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d46df-e980-445d-908f-29867ff10022 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.859894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d46df-e980-445d-908f-29867ff10022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:53.861875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d46df-e980-445d-908f-29867ff10022 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:53.893466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 965/1907] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:53 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:53.897993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce02057-8602-4eda-bbc5-24ece3a29bc6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:53.915261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d46df-e980-445d-908f-29867ff10022 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:53.917670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce02057-8602-4eda-bbc5-24ece3a29bc6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:53.917815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6c5d46df-e980-445d-908f-29867ff10022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.993483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:54.004459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d46df-e980-445d-908f-29867ff10022 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.009365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.010060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.010339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.010601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] PATH_INFO: `/v3/projects/bfc85d8bc0fb463e97503dc2b3a20390` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.011745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc85d8bc0fb463e97503dc2b3a20390)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.013720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.021591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85af5-0ff8-4a65-8111-89cc85c3a391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.022109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85af5-0ff8-4a65-8111-89cc85c3a391 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.022385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85af5-0ff8-4a65-8111-89cc85c3a391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.022653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85af5-0ff8-4a65-8111-89cc85c3a391 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.023287 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df85af5-0ff8-4a65-8111-89cc85c3a391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.025181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df85af5-0ff8-4a65-8111-89cc85c3a391 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.051298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 943/1908] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:53 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:54.052552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85af5-0ff8-4a65-8111-89cc85c3a391 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.063109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc85d8bc0fb463e97503dc2b3a20390', 'request_id': 'req-bea311e1-5ff2-4be9-a054-e2bf23964794'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.063941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85af5-0ff8-4a65-8111-89cc85c3a391 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.071101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc85d8bc0fb463e97503dc2b3a20390', 'request_id': 'req-bea311e1-5ff2-4be9-a054-e2bf23964794'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.081889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc85d8bc0fb463e97503dc2b3a20390', 'request_id': 'req-bea311e1-5ff2-4be9-a054-e2bf23964794'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.094575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] The token cache is being invalidate because project bfc85d8bc0fb463e97503dc2b3a20390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:54.095239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.095794 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.100782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 966/1909] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:53 2022] DELETE /identity/v3/projects/bfc85d8bc0fb463e97503dc2b3a20390 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:54.101751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.103456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:54.122399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28dde6-2580-498e-b207-f26d3b3912d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.123106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28dde6-2580-498e-b207-f26d3b3912d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.123459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28dde6-2580-498e-b207-f26d3b3912d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.123814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28dde6-2580-498e-b207-f26d3b3912d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.124528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d28dde6-2580-498e-b207-f26d3b3912d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.126997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d28dde6-2580-498e-b207-f26d3b3912d8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.129677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:54.141242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 944/1910] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:01:54.142365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28dde6-2580-498e-b207-f26d3b3912d8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.144077 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2d28dde6-2580-498e-b207-f26d3b3912d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:54.157843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28dde6-2580-498e-b207-f26d3b3912d8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.162829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39a274-f329-485a-b332-6a1e42bd9d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.163807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39a274-f329-485a-b332-6a1e42bd9d0d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.163927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39a274-f329-485a-b332-6a1e42bd9d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.164185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39a274-f329-485a-b332-6a1e42bd9d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.164927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39a274-f329-485a-b332-6a1e42bd9d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.166689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39a274-f329-485a-b332-6a1e42bd9d0d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.183053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 945/1911] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:01:54.184238 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39a274-f329-485a-b332-6a1e42bd9d0d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.188290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea311e1-5ff2-4be9-a054-e2bf23964794 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.193457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.194210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.194595 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.194957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] PATH_INFO: `/v3/users/c1b07978810a451e9e1c9309f487fb61` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.196252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1b07978810a451e9e1c9309f487fb61)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.196363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39a274-f329-485a-b332-6a1e42bd9d0d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.198450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.202261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1ee74-1948-4bd4-9300-83145e573428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.203813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ee74-1948-4bd4-9300-83145e573428 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.204096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ee74-1948-4bd4-9300-83145e573428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.205038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ee74-1948-4bd4-9300-83145e573428 admin admin] PATH_INFO: `/v3/projects/2c18e1da199c437b809a45aada0b0e0e/users/fed877b0a569455597dfbb8b98ae0bc0/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.227272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1ee74-1948-4bd4-9300-83145e573428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c18e1da199c437b809a45aada0b0e0e, user_id=fed877b0a569455597dfbb8b98ae0bc0, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.230382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1ee74-1948-4bd4-9300-83145e573428 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.254477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 946/1912] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:54 2022] PUT /identity/v3/projects/2c18e1da199c437b809a45aada0b0e0e/users/fed877b0a569455597dfbb8b98ae0bc0/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:54.256141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084240e-37ad-4e82-a78e-2b24855115ec None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.256490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084240e-37ad-4e82-a78e-2b24855115ec None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.256821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084240e-37ad-4e82-a78e-2b24855115ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.261748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1b07978810a451e9e1c9309f487fb61'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.270762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1b07978810a451e9e1c9309f487fb61'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.278423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1b07978810a451e9e1c9309f487fb61'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.279062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1b07978810a451e9e1c9309f487fb61'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.281567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6084240e-37ad-4e82-a78e-2b24855115ec None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:54.282795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6084240e-37ad-4e82-a78e-2b24855115ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:54.293915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1b07978810a451e9e1c9309f487fb61'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.301371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 967/1913] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:54 2022] DELETE /identity/v3/users/c1b07978810a451e9e1c9309f487fb61 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:54.347414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 947/1914] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:54.357656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.359827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:54.420561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de91e5d-1c8f-4d3a-8bfc-4a30b04c4250 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.425101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778b9ac-e83e-4d34-90f3-fd02e0ac8194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.425753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778b9ac-e83e-4d34-90f3-fd02e0ac8194 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.426031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778b9ac-e83e-4d34-90f3-fd02e0ac8194 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.426299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778b9ac-e83e-4d34-90f3-fd02e0ac8194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.501120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:54.512601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778b9ac-e83e-4d34-90f3-fd02e0ac8194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.514148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778b9ac-e83e-4d34-90f3-fd02e0ac8194 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.527404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 968/1915] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:54.596557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084240e-37ad-4e82-a78e-2b24855115ec None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.598010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6084240e-37ad-4e82-a78e-2b24855115ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:54.616786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084240e-37ad-4e82-a78e-2b24855115ec None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.628880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.629546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.629822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.630089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] PATH_INFO: `/v3/projects/e64abef6418b4f6c86e0ee09cbc39a7e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.631738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e64abef6418b4f6c86e0ee09cbc39a7e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.633924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.664909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:54.692559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e64abef6418b4f6c86e0ee09cbc39a7e', 'request_id': 'req-43aeee82-217f-4be2-8c45-48a2ce910f05'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.700101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e64abef6418b4f6c86e0ee09cbc39a7e', 'request_id': 'req-43aeee82-217f-4be2-8c45-48a2ce910f05'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.711087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e64abef6418b4f6c86e0ee09cbc39a7e', 'request_id': 'req-43aeee82-217f-4be2-8c45-48a2ce910f05'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.726944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] The token cache is being invalidate because project e64abef6418b4f6c86e0ee09cbc39a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:54.727352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.728164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-43aeee82-217f-4be2-8c45-48a2ce910f05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:54.731173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 948/1916] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:54 2022] DELETE /identity/v3/projects/e64abef6418b4f6c86e0ee09cbc39a7e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:54.735217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2efc99-9b3c-48a4-a659-fad808fc2b22 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.735624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2efc99-9b3c-48a4-a659-fad808fc2b22 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.736242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2efc99-9b3c-48a4-a659-fad808fc2b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.759256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ac2efc99-9b3c-48a4-a659-fad808fc2b22 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:54.760793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ac2efc99-9b3c-48a4-a659-fad808fc2b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:54.843748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 969/1917] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:01:54.845625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1db5-cbd8-435f-a415-78bba3303054 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.845980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1db5-cbd8-435f-a415-78bba3303054 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.846262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1db5-cbd8-435f-a415-78bba3303054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.867067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-945c1db5-cbd8-435f-a415-78bba3303054 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:54.868275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-945c1db5-cbd8-435f-a415-78bba3303054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:54.878917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 949/1918] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:54.883517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2efc99-9b3c-48a4-a659-fad808fc2b22 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.901026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2efc99-9b3c-48a4-a659-fad808fc2b22 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.907886 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54d7c2-49ac-4212-9b56-beed57e74482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.908534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d7c2-49ac-4212-9b56-beed57e74482 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.908817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d7c2-49ac-4212-9b56-beed57e74482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.909103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d7c2-49ac-4212-9b56-beed57e74482 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.909803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d7c2-49ac-4212-9b56-beed57e74482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:54.910107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d7c2-49ac-4212-9b56-beed57e74482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.911801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d7c2-49ac-4212-9b56-beed57e74482 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.940760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 970/1919] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:54.945567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1db5-cbd8-435f-a415-78bba3303054 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.957431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1db5-cbd8-435f-a415-78bba3303054 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.963357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e91aba-3a81-41d5-b709-5022383db23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.964307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e91aba-3a81-41d5-b709-5022383db23b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.964710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e91aba-3a81-41d5-b709-5022383db23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.965655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e91aba-3a81-41d5-b709-5022383db23b admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.965850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e91aba-3a81-41d5-b709-5022383db23b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.969547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e91aba-3a81-41d5-b709-5022383db23b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.024640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 950/1920] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:54 2022] POST /identity/v3/projects => generated 399 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:55.029653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54d7c2-49ac-4212-9b56-beed57e74482 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.043995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54d7c2-49ac-4212-9b56-beed57e74482 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.137514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a8776-d58d-45b7-845e-17dee60675a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.138967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a8776-d58d-45b7-845e-17dee60675a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.139775 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a8776-d58d-45b7-845e-17dee60675a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.140796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a8776-d58d-45b7-845e-17dee60675a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.141542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a8776-d58d-45b7-845e-17dee60675a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.144299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a8776-d58d-45b7-845e-17dee60675a6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.181961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 971/1921] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:55.186186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e91aba-3a81-41d5-b709-5022383db23b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.198493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e91aba-3a81-41d5-b709-5022383db23b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.204199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d0618-7ed6-4279-b085-e3e0fcd9cd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.204821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d0618-7ed6-4279-b085-e3e0fcd9cd14 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.205137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d0618-7ed6-4279-b085-e3e0fcd9cd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.205393 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d0618-7ed6-4279-b085-e3e0fcd9cd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.206047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d0618-7ed6-4279-b085-e3e0fcd9cd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.207653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d0618-7ed6-4279-b085-e3e0fcd9cd14 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.220083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 951/1922] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:55.224077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a8776-d58d-45b7-845e-17dee60675a6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.235082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a8776-d58d-45b7-845e-17dee60675a6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.241525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428aef9-3497-4c4d-b92f-6970a2cf0494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.242728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428aef9-3497-4c4d-b92f-6970a2cf0494 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.243028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428aef9-3497-4c4d-b92f-6970a2cf0494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.243338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428aef9-3497-4c4d-b92f-6970a2cf0494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.243985 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428aef9-3497-4c4d-b92f-6970a2cf0494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.246009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428aef9-3497-4c4d-b92f-6970a2cf0494 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.274748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 972/1923] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:55.275940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d0618-7ed6-4279-b085-e3e0fcd9cd14 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.287656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d0618-7ed6-4279-b085-e3e0fcd9cd14 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.297301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485d9e2-cb1f-479b-bb3a-9cb55abe7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.298084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485d9e2-cb1f-479b-bb3a-9cb55abe7a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.298331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485d9e2-cb1f-479b-bb3a-9cb55abe7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.298648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485d9e2-cb1f-479b-bb3a-9cb55abe7a17 admin admin] PATH_INFO: `/v3/projects/4428d7d5c4c7442582b432e02d2fbddb/users/57bd9182ceba4e869ceb6fb1240915aa/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.319335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485d9e2-cb1f-479b-bb3a-9cb55abe7a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4428d7d5c4c7442582b432e02d2fbddb, user_id=57bd9182ceba4e869ceb6fb1240915aa, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.322225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485d9e2-cb1f-479b-bb3a-9cb55abe7a17 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.335775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 952/1924] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:55 2022] PUT /identity/v3/projects/4428d7d5c4c7442582b432e02d2fbddb/users/57bd9182ceba4e869ceb6fb1240915aa/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:55.346398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b0dc9-5e51-4302-aaff-384a5cd5a64c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.346706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b0dc9-5e51-4302-aaff-384a5cd5a64c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.346978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b0dc9-5e51-4302-aaff-384a5cd5a64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.378233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-de3b0dc9-5e51-4302-aaff-384a5cd5a64c None None] MFA Rules not processed for user `57bd9182ceba4e869ceb6fb1240915aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:55.379510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-de3b0dc9-5e51-4302-aaff-384a5cd5a64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:55.422822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a147c1-9eb7-4943-aa9d-32dd63770fbb None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.423344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a147c1-9eb7-4943-aa9d-32dd63770fbb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.423898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a147c1-9eb7-4943-aa9d-32dd63770fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.440166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:55.465442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 973/1925] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:55.465648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-81a147c1-9eb7-4943-aa9d-32dd63770fbb None None] MFA Rules not processed for user `fed877b0a569455597dfbb8b98ae0bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:55.466962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-81a147c1-9eb7-4943-aa9d-32dd63770fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:55.470181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b0dc9-5e51-4302-aaff-384a5cd5a64c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.540803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b0dc9-5e51-4302-aaff-384a5cd5a64c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.545107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 953/1926] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:55.546211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222413fc-f4e3-45ac-b2be-089737970791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.547408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222413fc-f4e3-45ac-b2be-089737970791 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.547677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222413fc-f4e3-45ac-b2be-089737970791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.547870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222413fc-f4e3-45ac-b2be-089737970791 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.548526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222413fc-f4e3-45ac-b2be-089737970791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.550509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222413fc-f4e3-45ac-b2be-089737970791 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.556922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a147c1-9eb7-4943-aa9d-32dd63770fbb None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.558519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-81a147c1-9eb7-4943-aa9d-32dd63770fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:55.604589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 974/1927] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/projects => generated 399 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:55.605417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222413fc-f4e3-45ac-b2be-089737970791 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.611121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:55.635743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a147c1-9eb7-4943-aa9d-32dd63770fbb None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.682493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222413fc-f4e3-45ac-b2be-089737970791 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.688310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab748994-e898-44e9-8fcc-ec43c985ac81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.688932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748994-e898-44e9-8fcc-ec43c985ac81 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.689250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748994-e898-44e9-8fcc-ec43c985ac81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.689526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748994-e898-44e9-8fcc-ec43c985ac81 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.690236 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab748994-e898-44e9-8fcc-ec43c985ac81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.692177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab748994-e898-44e9-8fcc-ec43c985ac81 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.696304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1044b-ef0b-43db-9b16-782cddfa4cf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.696912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1044b-ef0b-43db-9b16-782cddfa4cf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.697221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1044b-ef0b-43db-9b16-782cddfa4cf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.697489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1044b-ef0b-43db-9b16-782cddfa4cf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.721472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 975/1928] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/users => generated 306 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:55.725395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab748994-e898-44e9-8fcc-ec43c985ac81 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.739287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab748994-e898-44e9-8fcc-ec43c985ac81 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.744644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e54bff-ee96-44bd-a484-853117b75484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.745328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e54bff-ee96-44bd-a484-853117b75484 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.745619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e54bff-ee96-44bd-a484-853117b75484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.745874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e54bff-ee96-44bd-a484-853117b75484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.746523 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e54bff-ee96-44bd-a484-853117b75484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.748145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e54bff-ee96-44bd-a484-853117b75484 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.762867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 976/1929] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:55.764082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e54bff-ee96-44bd-a484-853117b75484 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.775265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1044b-ef0b-43db-9b16-782cddfa4cf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.776737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1044b-ef0b-43db-9b16-782cddfa4cf4 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.778830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e54bff-ee96-44bd-a484-853117b75484 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.786434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4df6f2-d262-4aa4-be99-6efc8ec44958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.787269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4df6f2-d262-4aa4-be99-6efc8ec44958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.787605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4df6f2-d262-4aa4-be99-6efc8ec44958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.787996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4df6f2-d262-4aa4-be99-6efc8ec44958 admin admin] PATH_INFO: `/v3/projects/32f2feac56d34a4e890d9b2ae1a213d3/users/b4ce22804a524a598576543b057573f2/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.795504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 954/1930] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:01:55.812457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4df6f2-d262-4aa4-be99-6efc8ec44958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f2feac56d34a4e890d9b2ae1a213d3, user_id=b4ce22804a524a598576543b057573f2, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.815714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4df6f2-d262-4aa4-be99-6efc8ec44958 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.837051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 977/1931] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:55 2022] PUT /identity/v3/projects/32f2feac56d34a4e890d9b2ae1a213d3/users/b4ce22804a524a598576543b057573f2/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:55.841290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1044b-ef0b-43db-9b16-782cddfa4cf4 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.854550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1044b-ef0b-43db-9b16-782cddfa4cf4 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.916873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17236721-c98e-4e97-aa41-d069f3c95715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.917577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17236721-c98e-4e97-aa41-d069f3c95715 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.917853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17236721-c98e-4e97-aa41-d069f3c95715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.918200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17236721-c98e-4e97-aa41-d069f3c95715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.919109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17236721-c98e-4e97-aa41-d069f3c95715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.921909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17236721-c98e-4e97-aa41-d069f3c95715 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.933136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 955/1932] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:01:55.937587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4df6f2-d262-4aa4-be99-6efc8ec44958 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.949032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4df6f2-d262-4aa4-be99-6efc8ec44958 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.954799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d42433-c120-47de-adfa-728f49598ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.955566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d42433-c120-47de-adfa-728f49598ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.955876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d42433-c120-47de-adfa-728f49598ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.956145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d42433-c120-47de-adfa-728f49598ff1 admin admin] PATH_INFO: `/v3/projects/32f2feac56d34a4e890d9b2ae1a213d3/users/b4ce22804a524a598576543b057573f2/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.958288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d42433-c120-47de-adfa-728f49598ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f2feac56d34a4e890d9b2ae1a213d3, user_id=b4ce22804a524a598576543b057573f2, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.965791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d42433-c120-47de-adfa-728f49598ff1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.981345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 978/1933] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:55 2022] PUT /identity/v3/projects/32f2feac56d34a4e890d9b2ae1a213d3/users/b4ce22804a524a598576543b057573f2/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:55.994845 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce8c80-e81b-4cfe-b862-fa05624b10d1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.996008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce8c80-e81b-4cfe-b862-fa05624b10d1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.996857 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce8c80-e81b-4cfe-b862-fa05624b10d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:56.032984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-15ce8c80-e81b-4cfe-b862-fa05624b10d1 None None] MFA Rules not processed for user `b4ce22804a524a598576543b057573f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:56.035231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-15ce8c80-e81b-4cfe-b862-fa05624b10d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:56.086778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:56.165768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 956/1934] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:56.177297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d42433-c120-47de-adfa-728f49598ff1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:56.193305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:56.256420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d42433-c120-47de-adfa-728f49598ff1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:56.260099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd37056-a1f6-41a5-a2e3-5a22452a096b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:56.260711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd37056-a1f6-41a5-a2e3-5a22452a096b service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:56.260980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd37056-a1f6-41a5-a2e3-5a22452a096b service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:56.261276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd37056-a1f6-41a5-a2e3-5a22452a096b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:56.272023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd37056-a1f6-41a5-a2e3-5a22452a096b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:56.273509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd37056-a1f6-41a5-a2e3-5a22452a096b service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:56.286275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 979/1935] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:01:56.401013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce8c80-e81b-4cfe-b862-fa05624b10d1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:56.412837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce8c80-e81b-4cfe-b862-fa05624b10d1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:56.417980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669c7ba-bd70-4a80-95c8-85c515b3dafb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:56.418591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669c7ba-bd70-4a80-95c8-85c515b3dafb service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:56.418866 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669c7ba-bd70-4a80-95c8-85c515b3dafb service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:56.419138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669c7ba-bd70-4a80-95c8-85c515b3dafb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:56.503099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669c7ba-bd70-4a80-95c8-85c515b3dafb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:56.505304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669c7ba-bd70-4a80-95c8-85c515b3dafb service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:56.518790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 957/1936] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:57.310988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c3a11-8583-4141-8d35-dcea8a6cc8e7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:57.312524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c3a11-8583-4141-8d35-dcea8a6cc8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:57.312996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c3a11-8583-4141-8d35-dcea8a6cc8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:57.345674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ed9c3a11-8583-4141-8d35-dcea8a6cc8e7 None None] MFA Rules not processed for user `fed877b0a569455597dfbb8b98ae0bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:57.347026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ed9c3a11-8583-4141-8d35-dcea8a6cc8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:57.352944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669c7ba-bd70-4a80-95c8-85c515b3dafb service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:57.354988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f669c7ba-bd70-4a80-95c8-85c515b3dafb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:57.419058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 980/1937] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:57.434646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669c7ba-bd70-4a80-95c8-85c515b3dafb service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:57.441018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:57.441686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:57.441991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:57.442261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] PATH_INFO: `/v3/users/8baaf27dca57456786ae37d6ee4557f5` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:57.443412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8baaf27dca57456786ae37d6ee4557f5)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:57.445410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:57.498690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8baaf27dca57456786ae37d6ee4557f5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:57.510015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8baaf27dca57456786ae37d6ee4557f5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:57.510760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8baaf27dca57456786ae37d6ee4557f5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:57.511918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8baaf27dca57456786ae37d6ee4557f5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:57.527100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:57.529709 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8baaf27dca57456786ae37d6ee4557f5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:57.549386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 958/1938] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:57 2022] DELETE /identity/v3/users/8baaf27dca57456786ae37d6ee4557f5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:57.554244 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db1f25-9d4e-4e25-a49b-20138d87596c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:57.554531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db1f25-9d4e-4e25-a49b-20138d87596c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:57.554800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db1f25-9d4e-4e25-a49b-20138d87596c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:57.592783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-29db1f25-9d4e-4e25-a49b-20138d87596c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:57.594944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-29db1f25-9d4e-4e25-a49b-20138d87596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:57.671846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 981/1939] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:57.694458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:57.696578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:57.759371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc12b93-bc4e-4278-850b-3939b6b7d22c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:57.762968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe714d-b252-4deb-8ccf-d566623d16d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:57.763565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe714d-b252-4deb-8ccf-d566623d16d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:57.763840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe714d-b252-4deb-8ccf-d566623d16d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:57.764107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe714d-b252-4deb-8ccf-d566623d16d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:57.776581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe714d-b252-4deb-8ccf-d566623d16d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:57.778050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe714d-b252-4deb-8ccf-d566623d16d4 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:57.788199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 959/1940] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:58.285377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db1f25-9d4e-4e25-a49b-20138d87596c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:58.286880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-29db1f25-9d4e-4e25-a49b-20138d87596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:58.315361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db1f25-9d4e-4e25-a49b-20138d87596c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:58.333834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:58.341446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:58.341813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:58.342158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] PATH_INFO: `/v3/projects/7728131ac0f34701b17f8508a83c1e96` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:58.344350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7728131ac0f34701b17f8508a83c1e96)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:58.346231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:58.470347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7728131ac0f34701b17f8508a83c1e96', 'request_id': 'req-daff4362-5bc8-4f43-b7f9-724046d83431'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.479086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7728131ac0f34701b17f8508a83c1e96', 'request_id': 'req-daff4362-5bc8-4f43-b7f9-724046d83431'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.481275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:58.508558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7728131ac0f34701b17f8508a83c1e96', 'request_id': 'req-daff4362-5bc8-4f43-b7f9-724046d83431'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.525398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] The token cache is being invalidate because project 7728131ac0f34701b17f8508a83c1e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:58.525705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.526649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.528619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 982/1941] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:58 2022] DELETE /identity/v3/projects/7728131ac0f34701b17f8508a83c1e96 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:58.533838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe714d-b252-4deb-8ccf-d566623d16d4 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:58.551865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe714d-b252-4deb-8ccf-d566623d16d4 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:58.652369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:58.653168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:58.653550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:58.653922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] PATH_INFO: `/v3/users/947b770e17b840469f2dd86fbfe348b4` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:58.657171 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] RBAC: Authorizing `identity:delete_user(user_id=947b770e17b840469f2dd86fbfe348b4)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:58.659611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:58.764972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947b770e17b840469f2dd86fbfe348b4'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.775983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947b770e17b840469f2dd86fbfe348b4'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.776942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947b770e17b840469f2dd86fbfe348b4'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.777858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947b770e17b840469f2dd86fbfe348b4'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.799278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc5d5976-6e5e-4e31-8daf-918e9dbce37f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947b770e17b840469f2dd86fbfe348b4'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:58.818491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 960/1942] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:58 2022] DELETE /identity/v3/users/947b770e17b840469f2dd86fbfe348b4 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:59.338262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.339877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:59.355335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff4362-5bc8-4f43-b7f9-724046d83431 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.418385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.419284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.419885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.420268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] PATH_INFO: `/v3/projects/56487006d08942f996065d66c17e2fd7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.422249 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56487006d08942f996065d66c17e2fd7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.424298 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.520897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56487006d08942f996065d66c17e2fd7', 'request_id': 'req-afc507ff-d472-4b96-ab45-7dadf0f365f5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:59.533770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56487006d08942f996065d66c17e2fd7', 'request_id': 'req-afc507ff-d472-4b96-ab45-7dadf0f365f5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:59.550575 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56487006d08942f996065d66c17e2fd7', 'request_id': 'req-afc507ff-d472-4b96-ab45-7dadf0f365f5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:59.562510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] The token cache is being invalidate because project 56487006d08942f996065d66c17e2fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:59.563013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:59.563686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-afc507ff-d472-4b96-ab45-7dadf0f365f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:59.565261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 983/1943] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:59 2022] DELETE /identity/v3/projects/56487006d08942f996065d66c17e2fd7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:59.570528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417014a-4bb6-4f71-b66d-851957d605a6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.570990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417014a-4bb6-4f71-b66d-851957d605a6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.571204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417014a-4bb6-4f71-b66d-851957d605a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.594221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:59.602728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9417014a-4bb6-4f71-b66d-851957d605a6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:59.603529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9417014a-4bb6-4f71-b66d-851957d605a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:59.694534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 961/1944] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:59.704731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec469a8-3e59-47da-85b7-caf08a68007c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.705025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec469a8-3e59-47da-85b7-caf08a68007c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.705330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec469a8-3e59-47da-85b7-caf08a68007c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.734780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7ec469a8-3e59-47da-85b7-caf08a68007c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:59.736858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7ec469a8-3e59-47da-85b7-caf08a68007c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:59.751603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 984/1945] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:59.755508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417014a-4bb6-4f71-b66d-851957d605a6 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.769934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417014a-4bb6-4f71-b66d-851957d605a6 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.777115 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a8a9b-8909-45b2-9199-7569c104f957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.777553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a8a9b-8909-45b2-9199-7569c104f957 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.777839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a8a9b-8909-45b2-9199-7569c104f957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.778104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a8a9b-8909-45b2-9199-7569c104f957 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.778878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a8a9b-8909-45b2-9199-7569c104f957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:59.779187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a8a9b-8909-45b2-9199-7569c104f957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.783167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a8a9b-8909-45b2-9199-7569c104f957 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.801121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 962/1946] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:59.803072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec469a8-3e59-47da-85b7-caf08a68007c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.819670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec469a8-3e59-47da-85b7-caf08a68007c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.824414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27461bd-fb1e-4760-b2b6-92deb9dbd889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.825025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27461bd-fb1e-4760-b2b6-92deb9dbd889 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.825335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27461bd-fb1e-4760-b2b6-92deb9dbd889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.825665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27461bd-fb1e-4760-b2b6-92deb9dbd889 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.826237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27461bd-fb1e-4760-b2b6-92deb9dbd889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.828915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27461bd-fb1e-4760-b2b6-92deb9dbd889 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.866756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 985/1947] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:59 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:59.871449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a8a9b-8909-45b2-9199-7569c104f957 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.886143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a8a9b-8909-45b2-9199-7569c104f957 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.916388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27461bd-fb1e-4760-b2b6-92deb9dbd889 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.918099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d27461bd-fb1e-4760-b2b6-92deb9dbd889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:59.948960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a02b1-ee3f-4285-90ba-3775048d5569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.949693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a02b1-ee3f-4285-90ba-3775048d5569 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.950071 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a02b1-ee3f-4285-90ba-3775048d5569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.950543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a02b1-ee3f-4285-90ba-3775048d5569 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.951695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a02b1-ee3f-4285-90ba-3775048d5569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.953964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a02b1-ee3f-4285-90ba-3775048d5569 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.971243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:59.981651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 963/1948] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:59 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:59.982710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a02b1-ee3f-4285-90ba-3775048d5569 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.994059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a02b1-ee3f-4285-90ba-3775048d5569 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.997420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27461bd-fb1e-4760-b2b6-92deb9dbd889 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.001410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf8d1e-1f77-4d84-88e9-6a15068bb596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.002076 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf8d1e-1f77-4d84-88e9-6a15068bb596 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.002386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf8d1e-1f77-4d84-88e9-6a15068bb596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.002642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf8d1e-1f77-4d84-88e9-6a15068bb596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.003114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.003311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf8d1e-1f77-4d84-88e9-6a15068bb596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.005920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.005920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.005920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] PATH_INFO: `/v3/users/457ea559dd084aec86d2c46e0baba76d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.006340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf8d1e-1f77-4d84-88e9-6a15068bb596 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.006896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=457ea559dd084aec86d2c46e0baba76d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.009403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.021614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 964/1949] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:00.022682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf8d1e-1f77-4d84-88e9-6a15068bb596 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.038284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf8d1e-1f77-4d84-88e9-6a15068bb596 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.043762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d249040-cced-4a5d-b95d-e65148bbb06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.044470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d249040-cced-4a5d-b95d-e65148bbb06f admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.044844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d249040-cced-4a5d-b95d-e65148bbb06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.045249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d249040-cced-4a5d-b95d-e65148bbb06f admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.046296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d249040-cced-4a5d-b95d-e65148bbb06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.047921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d249040-cced-4a5d-b95d-e65148bbb06f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.061133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 965/1950] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:00.064986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d249040-cced-4a5d-b95d-e65148bbb06f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.069113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457ea559dd084aec86d2c46e0baba76d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:00.080569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d249040-cced-4a5d-b95d-e65148bbb06f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.082278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457ea559dd084aec86d2c46e0baba76d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:00.087998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1123e9-ed7c-461e-8f1b-5c2642f2520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.089301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1123e9-ed7c-461e-8f1b-5c2642f2520e admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.089741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1123e9-ed7c-461e-8f1b-5c2642f2520e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.090813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1123e9-ed7c-461e-8f1b-5c2642f2520e admin admin] PATH_INFO: `/v3/projects/e9e6db6621b948a18a31157ebaf6f7e1/users/3ccb8711b1ba4a3bb24068651686bfab/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.092662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457ea559dd084aec86d2c46e0baba76d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:00.095109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457ea559dd084aec86d2c46e0baba76d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:00.112842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1123e9-ed7c-461e-8f1b-5c2642f2520e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e6db6621b948a18a31157ebaf6f7e1, user_id=3ccb8711b1ba4a3bb24068651686bfab, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.117888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1123e9-ed7c-461e-8f1b-5c2642f2520e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.132220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2af56a70-2cda-415e-abd0-9aa3d435b0c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457ea559dd084aec86d2c46e0baba76d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:00.165487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 966/1951] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:00 2022] PUT /identity/v3/projects/e9e6db6621b948a18a31157ebaf6f7e1/users/3ccb8711b1ba4a3bb24068651686bfab/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:00.165689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 986/1952] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:59 2022] DELETE /identity/v3/users/457ea559dd084aec86d2c46e0baba76d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:00.165785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a6a6b-daa8-4ec4-9685-12de406ab5d3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.165785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a6a6b-daa8-4ec4-9685-12de406ab5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.165785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a6a6b-daa8-4ec4-9685-12de406ab5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.174616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f887b-e709-4b4f-9327-54fa1bc5f47a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.175031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f887b-e709-4b4f-9327-54fa1bc5f47a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.175290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f887b-e709-4b4f-9327-54fa1bc5f47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.219561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-579a6a6b-daa8-4ec4-9685-12de406ab5d3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:00.224121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-579a6a6b-daa8-4ec4-9685-12de406ab5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:00.229617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3d3f887b-e709-4b4f-9327-54fa1bc5f47a None None] MFA Rules not processed for user `3ccb8711b1ba4a3bb24068651686bfab`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:00.231687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3d3f887b-e709-4b4f-9327-54fa1bc5f47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:00.275281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:00.316804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 967/1953] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:00.326626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a6a6b-daa8-4ec4-9685-12de406ab5d3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.329005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-579a6a6b-daa8-4ec4-9685-12de406ab5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:00.351455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 987/1954] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:00.352464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f887b-e709-4b4f-9327-54fa1bc5f47a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.354185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3d3f887b-e709-4b4f-9327-54fa1bc5f47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:00.371222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f887b-e709-4b4f-9327-54fa1bc5f47a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.379652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f83ac1-b0a4-41a7-8584-e4fe41bd5eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.380284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f83ac1-b0a4-41a7-8584-e4fe41bd5eca admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.380564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f83ac1-b0a4-41a7-8584-e4fe41bd5eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.380836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f83ac1-b0a4-41a7-8584-e4fe41bd5eca admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.381549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f83ac1-b0a4-41a7-8584-e4fe41bd5eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.384513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f83ac1-b0a4-41a7-8584-e4fe41bd5eca admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.405257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a6a6b-daa8-4ec4-9685-12de406ab5d3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.410697 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dec0be-04f7-4637-9bfa-0d8d6fb96864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.411510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dec0be-04f7-4637-9bfa-0d8d6fb96864 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.411792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dec0be-04f7-4637-9bfa-0d8d6fb96864 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.412053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dec0be-04f7-4637-9bfa-0d8d6fb96864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.424990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dec0be-04f7-4637-9bfa-0d8d6fb96864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.426523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dec0be-04f7-4637-9bfa-0d8d6fb96864 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.439186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 988/1955] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:00 2022] POST /identity/v3/projects => generated 367 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:00.439341 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 968/1956] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:00.440672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f83ac1-b0a4-41a7-8584-e4fe41bd5eca admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.481229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:00.520536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f83ac1-b0a4-41a7-8584-e4fe41bd5eca admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.525552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1340d6-84fa-4d4d-8b60-85ee553d04d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.526264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1340d6-84fa-4d4d-8b60-85ee553d04d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.526607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1340d6-84fa-4d4d-8b60-85ee553d04d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.526961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1340d6-84fa-4d4d-8b60-85ee553d04d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.527656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1340d6-84fa-4d4d-8b60-85ee553d04d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.529708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1340d6-84fa-4d4d-8b60-85ee553d04d3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.558377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 989/1957] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:00 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:00.563488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dec0be-04f7-4637-9bfa-0d8d6fb96864 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.575650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dec0be-04f7-4637-9bfa-0d8d6fb96864 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.581063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25de7e2-92e3-4516-b80e-856bd435bfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.581779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25de7e2-92e3-4516-b80e-856bd435bfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.582269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25de7e2-92e3-4516-b80e-856bd435bfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.582550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25de7e2-92e3-4516-b80e-856bd435bfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.583204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25de7e2-92e3-4516-b80e-856bd435bfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.585292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25de7e2-92e3-4516-b80e-856bd435bfd0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.597934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 969/1958] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:00.604122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1340d6-84fa-4d4d-8b60-85ee553d04d3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.617693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1340d6-84fa-4d4d-8b60-85ee553d04d3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.627451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f95d0-1e42-44e4-9cdc-7757f756d7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.628133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f95d0-1e42-44e4-9cdc-7757f756d7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.628420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f95d0-1e42-44e4-9cdc-7757f756d7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.628696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f95d0-1e42-44e4-9cdc-7757f756d7e7 admin admin] PATH_INFO: `/v3/projects/ff043b59e5974d409e86988d01ba170d/users/4a78602544da4ab6b61b9293f847c9e5/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.652590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f95d0-1e42-44e4-9cdc-7757f756d7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff043b59e5974d409e86988d01ba170d, user_id=4a78602544da4ab6b61b9293f847c9e5, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.655249 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f95d0-1e42-44e4-9cdc-7757f756d7e7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.675280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 990/1959] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:00 2022] PUT /identity/v3/projects/ff043b59e5974d409e86988d01ba170d/users/4a78602544da4ab6b61b9293f847c9e5/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:00.679471 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25de7e2-92e3-4516-b80e-856bd435bfd0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.695605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25de7e2-92e3-4516-b80e-856bd435bfd0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.798458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed898-245f-4e7f-95dc-eeef5280d004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.799199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed898-245f-4e7f-95dc-eeef5280d004 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.799556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed898-245f-4e7f-95dc-eeef5280d004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.799832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed898-245f-4e7f-95dc-eeef5280d004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.800482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ed898-245f-4e7f-95dc-eeef5280d004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.802124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ed898-245f-4e7f-95dc-eeef5280d004 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.813557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 970/1960] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:00.817697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f95d0-1e42-44e4-9cdc-7757f756d7e7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.829899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f95d0-1e42-44e4-9cdc-7757f756d7e7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.837167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d449b9-acc3-4c1c-9d51-d6f0f3e52260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.838067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d449b9-acc3-4c1c-9d51-d6f0f3e52260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.838398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d449b9-acc3-4c1c-9d51-d6f0f3e52260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.838635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d449b9-acc3-4c1c-9d51-d6f0f3e52260 admin admin] PATH_INFO: `/v3/projects/ff043b59e5974d409e86988d01ba170d/users/4a78602544da4ab6b61b9293f847c9e5/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.840665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d449b9-acc3-4c1c-9d51-d6f0f3e52260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff043b59e5974d409e86988d01ba170d, user_id=4a78602544da4ab6b61b9293f847c9e5, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.844645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d449b9-acc3-4c1c-9d51-d6f0f3e52260 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.849038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed898-245f-4e7f-95dc-eeef5280d004 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:00.851394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-892ed898-245f-4e7f-95dc-eeef5280d004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:00.867419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 991/1961] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:00 2022] PUT /identity/v3/projects/ff043b59e5974d409e86988d01ba170d/users/4a78602544da4ab6b61b9293f847c9e5/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:00.868567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed898-245f-4e7f-95dc-eeef5280d004 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:00.872083 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8aed3-033a-4226-a3d8-35ef9f4f32ac None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.872440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8aed3-033a-4226-a3d8-35ef9f4f32ac None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.873021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8aed3-033a-4226-a3d8-35ef9f4f32ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.947001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3df8aed3-033a-4226-a3d8-35ef9f4f32ac None None] MFA Rules not processed for user `4a78602544da4ab6b61b9293f847c9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:00.949946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3df8aed3-033a-4226-a3d8-35ef9f4f32ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:00.977948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:00.978610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:00.978864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:00.979140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] PATH_INFO: `/v3/projects/0b22b91373e24bca8a52332a8d66d77c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:00.982694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b22b91373e24bca8a52332a8d66d77c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:00.985063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:00.997233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:01.076772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 992/1962] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:01.131300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8aed3-033a-4226-a3d8-35ef9f4f32ac None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:01.133396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3df8aed3-033a-4226-a3d8-35ef9f4f32ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:01.145987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b22b91373e24bca8a52332a8d66d77c', 'request_id': 'req-f3c98bfe-9913-472f-983f-2de4d0110a29'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.159741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b22b91373e24bca8a52332a8d66d77c', 'request_id': 'req-f3c98bfe-9913-472f-983f-2de4d0110a29'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.183702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b22b91373e24bca8a52332a8d66d77c', 'request_id': 'req-f3c98bfe-9913-472f-983f-2de4d0110a29'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.237906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] The token cache is being invalidate because project 0b22b91373e24bca8a52332a8d66d77c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:01.238222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.241117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.241117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 971/1963] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:00 2022] DELETE /identity/v3/projects/0b22b91373e24bca8a52332a8d66d77c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:01.242174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:01.249241 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:01.263834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8aed3-033a-4226-a3d8-35ef9f4f32ac None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:01.299013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:01.387621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:01.388362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:01.388638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:01.388814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:01.394062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c98bfe-9913-472f-983f-2de4d0110a29 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:01.397870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:01.412893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:01.413634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:01.413964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:01.414245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] PATH_INFO: `/v3/users/57bd9182ceba4e869ceb6fb1240915aa` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:01.415800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57bd9182ceba4e869ceb6fb1240915aa)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:01.422243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:01.497953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57bd9182ceba4e869ceb6fb1240915aa'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.537022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57bd9182ceba4e869ceb6fb1240915aa'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.537022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57bd9182ceba4e869ceb6fb1240915aa'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.537262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57bd9182ceba4e869ceb6fb1240915aa'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.591092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:01.591900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-40c73c16-893d-45c8-8938-9550f57e79e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57bd9182ceba4e869ceb6fb1240915aa'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.596094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:01.626709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 993/1964] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:01.627663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:01.629290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:01.641226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 972/1965] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:01 2022] DELETE /identity/v3/users/57bd9182ceba4e869ceb6fb1240915aa => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:01.646597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776fb59-e594-458f-8cc6-943288810547 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:01.646983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776fb59-e594-458f-8cc6-943288810547 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:01.647335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776fb59-e594-458f-8cc6-943288810547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:01.650672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5fee1-0c9b-4eb7-83c2-f5b4e8d4dc02 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:01.687611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6776fb59-e594-458f-8cc6-943288810547 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:01.692997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6776fb59-e594-458f-8cc6-943288810547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:01.730807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:01.736342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:01.738177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:01.739296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:01.739296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] PATH_INFO: `/v3/users/a26b41d5755c467caaa9a26cdd7340b2` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:01.740753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a26b41d5755c467caaa9a26cdd7340b2)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:01.743625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:01.831696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 973/1966] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:01.849445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776fb59-e594-458f-8cc6-943288810547 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:01.853675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6776fb59-e594-458f-8cc6-943288810547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:01.931456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a26b41d5755c467caaa9a26cdd7340b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.950719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a26b41d5755c467caaa9a26cdd7340b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.965832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a26b41d5755c467caaa9a26cdd7340b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.969199 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a26b41d5755c467caaa9a26cdd7340b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:01.992229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776fb59-e594-458f-8cc6-943288810547 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:01.995511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:02.000018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8a834-9145-474f-bd47-1e825a50ade4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:02.000643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8a834-9145-474f-bd47-1e825a50ade4 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:02.000919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a26b41d5755c467caaa9a26cdd7340b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:02.001384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8a834-9145-474f-bd47-1e825a50ade4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:02.001737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8a834-9145-474f-bd47-1e825a50ade4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:02.003835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9ef8a834-9145-474f-bd47-1e825a50ade4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:02.012552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 994/1967] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:01 2022] DELETE /identity/v3/users/a26b41d5755c467caaa9a26cdd7340b2 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:02.020898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8a834-9145-474f-bd47-1e825a50ade4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:02.022521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8a834-9145-474f-bd47-1e825a50ade4 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:02.034610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 974/1968] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:02.553299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:02.555369 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:02.684921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5131f-9a60-41ba-8a73-c259dacd7c75 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:02.692026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:02.692884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:02.693393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:02.693793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] PATH_INFO: `/v3/projects/c560627674ee4116ad7071cfab067859` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:02.695391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c560627674ee4116ad7071cfab067859)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:02.697677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:02.805551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c560627674ee4116ad7071cfab067859', 'request_id': 'req-0d3e7d26-96e0-4d76-904a-14ec5490056d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:02.819657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c560627674ee4116ad7071cfab067859', 'request_id': 'req-0d3e7d26-96e0-4d76-904a-14ec5490056d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:02.832599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c560627674ee4116ad7071cfab067859', 'request_id': 'req-0d3e7d26-96e0-4d76-904a-14ec5490056d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:02.861937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] The token cache is being invalidate because project c560627674ee4116ad7071cfab067859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:02.863393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:02.864681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0d3e7d26-96e0-4d76-904a-14ec5490056d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:02.867536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 995/1969] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:02 2022] DELETE /identity/v3/projects/c560627674ee4116ad7071cfab067859 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:02.872169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2beb810-1622-47bc-8492-954eed3e1d6b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:02.872695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2beb810-1622-47bc-8492-954eed3e1d6b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:02.873060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2beb810-1622-47bc-8492-954eed3e1d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:02.907341 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f2beb810-1622-47bc-8492-954eed3e1d6b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:02.908596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f2beb810-1622-47bc-8492-954eed3e1d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:02.996188 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 975/1970] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:02.997805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb466e-7c97-4031-938a-c6f4bee35fc7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:02.998092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb466e-7c97-4031-938a-c6f4bee35fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:02.998372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb466e-7c97-4031-938a-c6f4bee35fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.034335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-26bb466e-7c97-4031-938a-c6f4bee35fc7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:03.035754 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-26bb466e-7c97-4031-938a-c6f4bee35fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:03.045370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 996/1971] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:03.050794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2beb810-1622-47bc-8492-954eed3e1d6b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.065158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2beb810-1622-47bc-8492-954eed3e1d6b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:03.089721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8f511-faf4-4431-8cc0-fc92294d6e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:03.090985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f511-faf4-4431-8cc0-fc92294d6e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:03.091256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f511-faf4-4431-8cc0-fc92294d6e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:03.091530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f511-faf4-4431-8cc0-fc92294d6e8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.092195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8f511-faf4-4431-8cc0-fc92294d6e8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:03.092496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8f511-faf4-4431-8cc0-fc92294d6e8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:03.096547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8f511-faf4-4431-8cc0-fc92294d6e8c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:03.112995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 976/1972] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:03.117815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb466e-7c97-4031-938a-c6f4bee35fc7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.129811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb466e-7c97-4031-938a-c6f4bee35fc7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:03.134514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a761e-126f-4b90-ae23-c5f73e14dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:03.135156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a761e-126f-4b90-ae23-c5f73e14dce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:03.135411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a761e-126f-4b90-ae23-c5f73e14dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:03.135685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a761e-126f-4b90-ae23-c5f73e14dce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.136322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a761e-126f-4b90-ae23-c5f73e14dce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:03.138264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a761e-126f-4b90-ae23-c5f73e14dce2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:03.169964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 997/1973] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:03 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:03.175032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8f511-faf4-4431-8cc0-fc92294d6e8c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.191269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8f511-faf4-4431-8cc0-fc92294d6e8c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:03.278661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:03.302839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4413d-1872-4d69-b547-9558abc0baef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:03.303458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4413d-1872-4d69-b547-9558abc0baef admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:03.303742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4413d-1872-4d69-b547-9558abc0baef admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:03.304015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4413d-1872-4d69-b547-9558abc0baef admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.304651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4413d-1872-4d69-b547-9558abc0baef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:03.306600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4413d-1872-4d69-b547-9558abc0baef admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:03.318225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a761e-126f-4b90-ae23-c5f73e14dce2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.321399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-310a761e-126f-4b90-ae23-c5f73e14dce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:03.346355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a761e-126f-4b90-ae23-c5f73e14dce2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:03.351610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 977/1974] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:03 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:03.352329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:03.353521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:03.354177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:03.354830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] PATH_INFO: `/v3/projects/4428d7d5c4c7442582b432e02d2fbddb` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.355775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4413d-1872-4d69-b547-9558abc0baef admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.356383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4428d7d5c4c7442582b432e02d2fbddb)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:03.360486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:03.369235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4413d-1872-4d69-b547-9558abc0baef admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:03.377896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6c280-fd55-4e7a-b4d2-717021a6d98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:03.378327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6c280-fd55-4e7a-b4d2-717021a6d98a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:03.378610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6c280-fd55-4e7a-b4d2-717021a6d98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:03.378883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6c280-fd55-4e7a-b4d2-717021a6d98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.379592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6c280-fd55-4e7a-b4d2-717021a6d98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:03.381137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6c280-fd55-4e7a-b4d2-717021a6d98a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:03.403919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 978/1975] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:03.406206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6c280-fd55-4e7a-b4d2-717021a6d98a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.420241 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6c280-fd55-4e7a-b4d2-717021a6d98a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:03.428760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f809896-8ae3-4412-9440-cb4c67bea9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:03.429656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f809896-8ae3-4412-9440-cb4c67bea9dd admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:03.430130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f809896-8ae3-4412-9440-cb4c67bea9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:03.430998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f809896-8ae3-4412-9440-cb4c67bea9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.432044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f809896-8ae3-4412-9440-cb4c67bea9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:03.434171 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f809896-8ae3-4412-9440-cb4c67bea9dd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:03.461956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 979/1976] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:03.466845 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f809896-8ae3-4412-9440-cb4c67bea9dd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.483088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f809896-8ae3-4412-9440-cb4c67bea9dd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:03.501806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4428d7d5c4c7442582b432e02d2fbddb', 'request_id': 'req-116f73f0-61fb-4afe-88c5-0ac6aa080528'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.503141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:03.518143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4428d7d5c4c7442582b432e02d2fbddb', 'request_id': 'req-116f73f0-61fb-4afe-88c5-0ac6aa080528'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.546863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4428d7d5c4c7442582b432e02d2fbddb', 'request_id': 'req-116f73f0-61fb-4afe-88c5-0ac6aa080528'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.575959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] The token cache is being invalidate because project 4428d7d5c4c7442582b432e02d2fbddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:03.576519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.577093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.594390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022e4a3-e24d-41c8-a548-da57aef28b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:03.595764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022e4a3-e24d-41c8-a548-da57aef28b07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:03.596121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022e4a3-e24d-41c8-a548-da57aef28b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:03.596330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 998/1977] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:03 2022] DELETE /identity/v3/projects/4428d7d5c4c7442582b432e02d2fbddb => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:03.601988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022e4a3-e24d-41c8-a548-da57aef28b07 admin admin] PATH_INFO: `/v3/projects/75874574a3f346b78a58218225f2e1b7/users/e9b5e1c1d85a4bd2b75d13bfaa7cc52d/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.602960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.604564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:03.621862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022e4a3-e24d-41c8-a548-da57aef28b07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75874574a3f346b78a58218225f2e1b7, user_id=e9b5e1c1d85a4bd2b75d13bfaa7cc52d, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:03.627439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022e4a3-e24d-41c8-a548-da57aef28b07 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:03.632380 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f73f0-61fb-4afe-88c5-0ac6aa080528 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:03.647419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:03.647419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:03.647419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:03.647419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] PATH_INFO: `/v3/users/b4ce22804a524a598576543b057573f2` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.647977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ce22804a524a598576543b057573f2)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:03.648706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:03.649981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 980/1978] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:03 2022] PUT /identity/v3/projects/75874574a3f346b78a58218225f2e1b7/users/e9b5e1c1d85a4bd2b75d13bfaa7cc52d/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:03.664616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d8ade-8360-448b-bf00-4ea1cb5baf71 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:03.666557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d8ade-8360-448b-bf00-4ea1cb5baf71 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:03.671749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d8ade-8360-448b-bf00-4ea1cb5baf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:03.735957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ce22804a524a598576543b057573f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.748590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e42d8ade-8360-448b-bf00-4ea1cb5baf71 None None] MFA Rules not processed for user `e9b5e1c1d85a4bd2b75d13bfaa7cc52d`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:03.751155 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e42d8ade-8360-448b-bf00-4ea1cb5baf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:03.760599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ce22804a524a598576543b057573f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.772281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ce22804a524a598576543b057573f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.774206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ce22804a524a598576543b057573f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.810480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ce22804a524a598576543b057573f2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:03.829906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 999/1979] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:03 2022] DELETE /identity/v3/users/b4ce22804a524a598576543b057573f2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:03.965360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.967009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:03.970701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 981/1980] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:03.982461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d8ade-8360-448b-bf00-4ea1cb5baf71 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:03.988072 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e42d8ade-8360-448b-bf00-4ea1cb5baf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:04.064700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:04.128519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c87392-b87e-430f-90f1-f9961da40902 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:04.135958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:04.136842 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:04.137430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:04.137941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] PATH_INFO: `/v3/users/3ccb8711b1ba4a3bb24068651686bfab` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:04.139308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ccb8711b1ba4a3bb24068651686bfab)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:04.142166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:04.209915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d8ade-8360-448b-bf00-4ea1cb5baf71 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:04.278709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ccb8711b1ba4a3bb24068651686bfab'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.290947 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=76,keystone:INSERT=3,keystone:DELETE=21 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:04.297390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ccb8711b1ba4a3bb24068651686bfab'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.311428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ccb8711b1ba4a3bb24068651686bfab'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.315110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ccb8711b1ba4a3bb24068651686bfab'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.348010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ef1983aa-2bec-4ad9-8f6e-ca7012e0ebac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ccb8711b1ba4a3bb24068651686bfab'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.356876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a47e77cc1942618a3584862042c5f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:04.357957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:04.359234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:04.359725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:04.362107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1000/1981] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:03 2022] DELETE /identity/v3/users/3ccb8711b1ba4a3bb24068651686bfab => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:04.363514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:04.367877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503e1e5-aa15-4e45-810b-1f71ce059966 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:04.368177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503e1e5-aa15-4e45-810b-1f71ce059966 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:04.368437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503e1e5-aa15-4e45-810b-1f71ce059966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:04.411324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-2503e1e5-aa15-4e45-810b-1f71ce059966 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:04.417460 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2503e1e5-aa15-4e45-810b-1f71ce059966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:04.509574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:04.511416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:04.538145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 982/1982] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:04.553921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1001/1983] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:04.578002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:04.580148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:04.692835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503e1e5-aa15-4e45-810b-1f71ce059966 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:04.693395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adb889-f2fd-46b0-a9d0-e6cc1ecc818c service glance] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:04.694528 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-2503e1e5-aa15-4e45-810b-1f71ce059966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:04.717115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503e1e5-aa15-4e45-810b-1f71ce059966 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:04.737625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:04.738039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:04.738323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:04.738648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] PATH_INFO: `/v3/projects/32f2feac56d34a4e890d9b2ae1a213d3` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:04.740958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:04.742103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f2feac56d34a4e890d9b2ae1a213d3)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:04.750074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:04.770720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcf6f3-144a-463e-bc77-e9ec66a4d0af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:04.771633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf6f3-144a-463e-bc77-e9ec66a4d0af service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:04.772001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf6f3-144a-463e-bc77-e9ec66a4d0af service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:04.772443 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf6f3-144a-463e-bc77-e9ec66a4d0af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:04.796939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcf6f3-144a-463e-bc77-e9ec66a4d0af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:04.798504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcf6f3-144a-463e-bc77-e9ec66a4d0af service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:04.826569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 983/1984] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:04.840829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f2feac56d34a4e890d9b2ae1a213d3', 'request_id': 'req-04328ce0-1b5b-41aa-b6d1-5ac498208386'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.849035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f2feac56d34a4e890d9b2ae1a213d3', 'request_id': 'req-04328ce0-1b5b-41aa-b6d1-5ac498208386'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.866203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f2feac56d34a4e890d9b2ae1a213d3', 'request_id': 'req-04328ce0-1b5b-41aa-b6d1-5ac498208386'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.915265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] The token cache is being invalidate because project 32f2feac56d34a4e890d9b2ae1a213d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:04.916822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.928571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-04328ce0-1b5b-41aa-b6d1-5ac498208386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:04.930545 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1002/1985] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:04 2022] DELETE /identity/v3/projects/32f2feac56d34a4e890d9b2ae1a213d3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:04.937114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecefc9a-6926-45d3-a6d6-46426d1e912b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:04.938947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecefc9a-6926-45d3-a6d6-46426d1e912b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:04.939462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecefc9a-6926-45d3-a6d6-46426d1e912b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:04.970872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-aecefc9a-6926-45d3-a6d6-46426d1e912b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:04.972113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-aecefc9a-6926-45d3-a6d6-46426d1e912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:05.099346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 984/1986] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:05.108283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03126787-5af1-4529-9dea-0d9c99dae6d2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.109176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03126787-5af1-4529-9dea-0d9c99dae6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.109467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03126787-5af1-4529-9dea-0d9c99dae6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.159738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-03126787-5af1-4529-9dea-0d9c99dae6d2 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:05.160978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-03126787-5af1-4529-9dea-0d9c99dae6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:05.190958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1003/1987] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:05.195238 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecefc9a-6926-45d3-a6d6-46426d1e912b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.208122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecefc9a-6926-45d3-a6d6-46426d1e912b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.232787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8040f-e580-48e7-9e08-ccb09721cf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.234913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8040f-e580-48e7-9e08-ccb09721cf87 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.236699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8040f-e580-48e7-9e08-ccb09721cf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.237688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8040f-e580-48e7-9e08-ccb09721cf87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.238391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8040f-e580-48e7-9e08-ccb09721cf87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:05.238702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8040f-e580-48e7-9e08-ccb09721cf87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:05.242425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8040f-e580-48e7-9e08-ccb09721cf87 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:05.263710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 985/1988] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:05.269952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03126787-5af1-4529-9dea-0d9c99dae6d2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.293737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03126787-5af1-4529-9dea-0d9c99dae6d2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.293737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4bd8c-88f4-4141-9839-e12ca00c4303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.293737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4bd8c-88f4-4141-9839-e12ca00c4303 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.293737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4bd8c-88f4-4141-9839-e12ca00c4303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.294288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4bd8c-88f4-4141-9839-e12ca00c4303 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.294288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4bd8c-88f4-4141-9839-e12ca00c4303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:05.294288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4bd8c-88f4-4141-9839-e12ca00c4303 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:05.372805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1004/1989] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:05 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:05.378218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8040f-e580-48e7-9e08-ccb09721cf87 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.410618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8040f-e580-48e7-9e08-ccb09721cf87 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.466431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4bd8c-88f4-4141-9839-e12ca00c4303 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.470965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d9f4bd8c-88f4-4141-9839-e12ca00c4303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:05.548350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:05.583341 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab4b8-b27c-4315-a694-43a626fcf1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.584473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab4b8-b27c-4315-a694-43a626fcf1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.585533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab4b8-b27c-4315-a694-43a626fcf1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.585822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab4b8-b27c-4315-a694-43a626fcf1c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.586632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ab4b8-b27c-4315-a694-43a626fcf1c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:05.590516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ab4b8-b27c-4315-a694-43a626fcf1c9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:05.631651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4bd8c-88f4-4141-9839-e12ca00c4303 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.633922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 986/1990] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:05 2022] POST /identity/v3/users => generated 294 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:05.637946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab4b8-b27c-4315-a694-43a626fcf1c9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.643213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.643889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.644164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.644430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] PATH_INFO: `/v3/projects/e9e6db6621b948a18a31157ebaf6f7e1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.645734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e6db6621b948a18a31157ebaf6f7e1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:05.647882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:05.661538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab4b8-b27c-4315-a694-43a626fcf1c9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.674726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523bb28-8b26-4104-91c0-37609fd28ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.675290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523bb28-8b26-4104-91c0-37609fd28ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.675636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523bb28-8b26-4104-91c0-37609fd28ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.675968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523bb28-8b26-4104-91c0-37609fd28ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.676684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523bb28-8b26-4104-91c0-37609fd28ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:05.678394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523bb28-8b26-4104-91c0-37609fd28ad7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:05.714951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 987/1991] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:05.717252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523bb28-8b26-4104-91c0-37609fd28ad7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.740120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e6db6621b948a18a31157ebaf6f7e1', 'request_id': 'req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:05.742462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523bb28-8b26-4104-91c0-37609fd28ad7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.759896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e6db6621b948a18a31157ebaf6f7e1', 'request_id': 'req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:05.780355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e6db6621b948a18a31157ebaf6f7e1', 'request_id': 'req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:05.794430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] The token cache is being invalidate because project e9e6db6621b948a18a31157ebaf6f7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:05.794820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:05.796082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:05.798257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1005/1992] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:05 2022] DELETE /identity/v3/projects/e9e6db6621b948a18a31157ebaf6f7e1 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:05.804233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.808648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:05.862517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:05.889009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca73122-f1dd-4440-b607-9289eebdab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.889642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca73122-f1dd-4440-b607-9289eebdab67 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.890057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca73122-f1dd-4440-b607-9289eebdab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.890340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca73122-f1dd-4440-b607-9289eebdab67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.891764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca73122-f1dd-4440-b607-9289eebdab67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:05.893726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca73122-f1dd-4440-b607-9289eebdab67 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:05.910437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 988/1993] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:05.914441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca73122-f1dd-4440-b607-9289eebdab67 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.917139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-dca73122-f1dd-4440-b607-9289eebdab67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:05.941258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca73122-f1dd-4440-b607-9289eebdab67 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.949449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b2391-3ea4-40a9-9ee1-fd852d77ffb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.950774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b2391-3ea4-40a9-9ee1-fd852d77ffb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.951187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b2391-3ea4-40a9-9ee1-fd852d77ffb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.951831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b2391-3ea4-40a9-9ee1-fd852d77ffb8 admin admin] PATH_INFO: `/v3/projects/b43c4817974a4d7f85dedbc102c083ea/users/668d7542b2dc4e11bceb626aad3c5b02/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.959409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4558e31-e6cd-4b25-af9f-fe54e4f8324f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.969844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.971505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.971926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.972271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] PATH_INFO: `/v3/users/4a78602544da4ab6b61b9293f847c9e5` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.973576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b2391-3ea4-40a9-9ee1-fd852d77ffb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43c4817974a4d7f85dedbc102c083ea, user_id=668d7542b2dc4e11bceb626aad3c5b02, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:05.974040 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a78602544da4ab6b61b9293f847c9e5)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:05.976449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:05.977288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b2391-3ea4-40a9-9ee1-fd852d77ffb8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.025106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 989/1994] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:05 2022] PUT /identity/v3/projects/b43c4817974a4d7f85dedbc102c083ea/users/668d7542b2dc4e11bceb626aad3c5b02/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:06.042939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11174dfc-ed13-4c04-a5db-402d02b80247 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.042939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11174dfc-ed13-4c04-a5db-402d02b80247 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.043949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11174dfc-ed13-4c04-a5db-402d02b80247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.116066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-11174dfc-ed13-4c04-a5db-402d02b80247 None None] MFA Rules not processed for user `668d7542b2dc4e11bceb626aad3c5b02`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:06.118264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-11174dfc-ed13-4c04-a5db-402d02b80247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:06.125140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a78602544da4ab6b61b9293f847c9e5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:06.141414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a78602544da4ab6b61b9293f847c9e5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:06.154279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.166686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a78602544da4ab6b61b9293f847c9e5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:06.167964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a78602544da4ab6b61b9293f847c9e5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:06.191464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a78602544da4ab6b61b9293f847c9e5'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:06.210801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1006/1995] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:05 2022] DELETE /identity/v3/users/4a78602544da4ab6b61b9293f847c9e5 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:06.265127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 990/1996] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:06.272555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.272555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:06.351049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345f795-a943-4b0d-937a-ea8c02f297ca admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.358175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d0968-ba3f-4eda-b88b-fa37f1b9c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.360842 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d0968-ba3f-4eda-b88b-fa37f1b9c9ec admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.361409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d0968-ba3f-4eda-b88b-fa37f1b9c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.362045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d0968-ba3f-4eda-b88b-fa37f1b9c9ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.363124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d0968-ba3f-4eda-b88b-fa37f1b9c9ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.366742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d0968-ba3f-4eda-b88b-fa37f1b9c9ec admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.419489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1007/1997] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:06 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:06.424507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11174dfc-ed13-4c04-a5db-402d02b80247 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.497582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11174dfc-ed13-4c04-a5db-402d02b80247 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.552382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.565587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468be7fa-2448-48eb-b1f6-2f6604b6dc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.566351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468be7fa-2448-48eb-b1f6-2f6604b6dc5c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.566696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468be7fa-2448-48eb-b1f6-2f6604b6dc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.567037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468be7fa-2448-48eb-b1f6-2f6604b6dc5c admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.567883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468be7fa-2448-48eb-b1f6-2f6604b6dc5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.570046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468be7fa-2448-48eb-b1f6-2f6604b6dc5c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.599448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 991/1998] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:06 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:06.601046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d0968-ba3f-4eda-b88b-fa37f1b9c9ec admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.626328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d0968-ba3f-4eda-b88b-fa37f1b9c9ec admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.635383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af425c6f-3bdc-403d-ac25-e3a204f70f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.636148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af425c6f-3bdc-403d-ac25-e3a204f70f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.636767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af425c6f-3bdc-403d-ac25-e3a204f70f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.636767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af425c6f-3bdc-403d-ac25-e3a204f70f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.637765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af425c6f-3bdc-403d-ac25-e3a204f70f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.640066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af425c6f-3bdc-403d-ac25-e3a204f70f05 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.664807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1008/1999] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:06.665680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468be7fa-2448-48eb-b1f6-2f6604b6dc5c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.680311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468be7fa-2448-48eb-b1f6-2f6604b6dc5c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.689609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61b6b5-26ab-49ac-9898-582c82567172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.690954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61b6b5-26ab-49ac-9898-582c82567172 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.692185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61b6b5-26ab-49ac-9898-582c82567172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.692925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61b6b5-26ab-49ac-9898-582c82567172 admin admin] PATH_INFO: `/v3/projects/2b662abf767c490d899d86cb6cd66bf7/users/e01dcd648c044b57824959b59723b4b2/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.713617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61b6b5-26ab-49ac-9898-582c82567172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b662abf767c490d899d86cb6cd66bf7, user_id=e01dcd648c044b57824959b59723b4b2, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.717172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61b6b5-26ab-49ac-9898-582c82567172 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.734728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 992/2000] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:06 2022] PUT /identity/v3/projects/2b662abf767c490d899d86cb6cd66bf7/users/e01dcd648c044b57824959b59723b4b2/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:06.738718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af425c6f-3bdc-403d-ac25-e3a204f70f05 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.751166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af425c6f-3bdc-403d-ac25-e3a204f70f05 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.833568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908749e8-8b8b-4ea3-b5d3-c630e285141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.834207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908749e8-8b8b-4ea3-b5d3-c630e285141b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.834488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908749e8-8b8b-4ea3-b5d3-c630e285141b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.834908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908749e8-8b8b-4ea3-b5d3-c630e285141b admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.836010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908749e8-8b8b-4ea3-b5d3-c630e285141b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.837562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908749e8-8b8b-4ea3-b5d3-c630e285141b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.852897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1009/2001] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:06.855141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61b6b5-26ab-49ac-9898-582c82567172 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.872409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61b6b5-26ab-49ac-9898-582c82567172 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.879502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12a191-76a9-410c-bcb2-998e9da0e4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.879502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12a191-76a9-410c-bcb2-998e9da0e4f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.879705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12a191-76a9-410c-bcb2-998e9da0e4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.879960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12a191-76a9-410c-bcb2-998e9da0e4f2 admin admin] PATH_INFO: `/v3/projects/2b662abf767c490d899d86cb6cd66bf7/users/e01dcd648c044b57824959b59723b4b2/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.894176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12a191-76a9-410c-bcb2-998e9da0e4f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b662abf767c490d899d86cb6cd66bf7, user_id=e01dcd648c044b57824959b59723b4b2, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.898460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12a191-76a9-410c-bcb2-998e9da0e4f2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.932105 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 993/2002] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:06 2022] PUT /identity/v3/projects/2b662abf767c490d899d86cb6cd66bf7/users/e01dcd648c044b57824959b59723b4b2/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:06.943140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0596881-212f-4e80-988b-24723d460538 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.945991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0596881-212f-4e80-988b-24723d460538 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.946492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0596881-212f-4e80-988b-24723d460538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.981048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f0596881-212f-4e80-988b-24723d460538 None None] MFA Rules not processed for user `e01dcd648c044b57824959b59723b4b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:06.982623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f0596881-212f-4e80-988b-24723d460538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:07.023038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.087469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1010/2003] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:07.096839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12a191-76a9-410c-bcb2-998e9da0e4f2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.101723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ab12a191-76a9-410c-bcb2-998e9da0e4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:07.215793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12a191-76a9-410c-bcb2-998e9da0e4f2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.225819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc326b3-70fd-4e86-b5e5-c19734eff68b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.227501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc326b3-70fd-4e86-b5e5-c19734eff68b service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.227784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc326b3-70fd-4e86-b5e5-c19734eff68b service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.228055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc326b3-70fd-4e86-b5e5-c19734eff68b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.264503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc326b3-70fd-4e86-b5e5-c19734eff68b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.270521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc326b3-70fd-4e86-b5e5-c19734eff68b service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.282987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0596881-212f-4e80-988b-24723d460538 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.285181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f0596881-212f-4e80-988b-24723d460538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:07.301156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 994/2004] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:07.391213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0596881-212f-4e80-988b-24723d460538 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.400637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.401530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.401841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.402224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] PATH_INFO: `/v3/projects/ff043b59e5974d409e86988d01ba170d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.403786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff043b59e5974d409e86988d01ba170d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.406414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.488366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff043b59e5974d409e86988d01ba170d', 'request_id': 'req-29c5bb0a-3053-429d-8926-b9d751aa04a7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.502101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff043b59e5974d409e86988d01ba170d', 'request_id': 'req-29c5bb0a-3053-429d-8926-b9d751aa04a7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.517263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff043b59e5974d409e86988d01ba170d', 'request_id': 'req-29c5bb0a-3053-429d-8926-b9d751aa04a7'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.533244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc326b3-70fd-4e86-b5e5-c19734eff68b service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.549233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] The token cache is being invalidate because project ff043b59e5974d409e86988d01ba170d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:07.549588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.549894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc326b3-70fd-4e86-b5e5-c19734eff68b service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.550195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-29c5bb0a-3053-429d-8926-b9d751aa04a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.551996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1011/2005] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:07 2022] DELETE /identity/v3/projects/ff043b59e5974d409e86988d01ba170d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:07.557225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9515d-490d-4db1-accd-af0626e84b26 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.557632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9515d-490d-4db1-accd-af0626e84b26 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.558013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9515d-490d-4db1-accd-af0626e84b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.593290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-47f9515d-490d-4db1-accd-af0626e84b26 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:07.594648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-47f9515d-490d-4db1-accd-af0626e84b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:07.633638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.634604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.634888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.635316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] PATH_INFO: `/v3/users/668d7542b2dc4e11bceb626aad3c5b02` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.637058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=668d7542b2dc4e11bceb626aad3c5b02)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.639028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.700391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.718409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1012/2006] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:07.728501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba61fd9-8a90-482e-82fc-a172d06b5830 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.729105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba61fd9-8a90-482e-82fc-a172d06b5830 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.729539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba61fd9-8a90-482e-82fc-a172d06b5830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.734454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '668d7542b2dc4e11bceb626aad3c5b02'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.736492 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.745846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '668d7542b2dc4e11bceb626aad3c5b02'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.747208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '668d7542b2dc4e11bceb626aad3c5b02'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.748389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '668d7542b2dc4e11bceb626aad3c5b02'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.759514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-dba61fd9-8a90-482e-82fc-a172d06b5830 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:07.761025 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-dba61fd9-8a90-482e-82fc-a172d06b5830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:07.769776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13a80ad1-c61c-4546-b1cf-1e34300ba914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '668d7542b2dc4e11bceb626aad3c5b02'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:07.775443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1013/2007] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:07.777055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba61fd9-8a90-482e-82fc-a172d06b5830 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.786423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba61fd9-8a90-482e-82fc-a172d06b5830 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.788676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 995/2008] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:07 2022] DELETE /identity/v3/users/668d7542b2dc4e11bceb626aad3c5b02 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:07.793602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643aae7b-0832-417d-82e0-ff927293d48e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.793845 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643aae7b-0832-417d-82e0-ff927293d48e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.794133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643aae7b-0832-417d-82e0-ff927293d48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.798831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2c4df-229e-4f0e-b2bd-06ec9233b824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.798831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2c4df-229e-4f0e-b2bd-06ec9233b824 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.798831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2c4df-229e-4f0e-b2bd-06ec9233b824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.798831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2c4df-229e-4f0e-b2bd-06ec9233b824 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.799431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2c4df-229e-4f0e-b2bd-06ec9233b824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:07.799928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2c4df-229e-4f0e-b2bd-06ec9233b824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.802417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2c4df-229e-4f0e-b2bd-06ec9233b824 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.817947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-643aae7b-0832-417d-82e0-ff927293d48e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:07.819190 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-643aae7b-0832-417d-82e0-ff927293d48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:07.823019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1014/2009] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:07.829698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2c4df-229e-4f0e-b2bd-06ec9233b824 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.845061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2c4df-229e-4f0e-b2bd-06ec9233b824 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.893956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 996/2010] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:07.904759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643aae7b-0832-417d-82e0-ff927293d48e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.906487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-643aae7b-0832-417d-82e0-ff927293d48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:07.920585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09736847-eabc-44fd-902c-e4cef4841f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.922005 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09736847-eabc-44fd-902c-e4cef4841f5b admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.922292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09736847-eabc-44fd-902c-e4cef4841f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.922564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09736847-eabc-44fd-902c-e4cef4841f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.923229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09736847-eabc-44fd-902c-e4cef4841f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.925167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09736847-eabc-44fd-902c-e4cef4841f5b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.956816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1015/2011] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:07 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:07.960620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09736847-eabc-44fd-902c-e4cef4841f5b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.975686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643aae7b-0832-417d-82e0-ff927293d48e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.038440 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac6b27-fd98-423b-95dc-454e88180a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.039064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6b27-fd98-423b-95dc-454e88180a3c service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.039390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6b27-fd98-423b-95dc-454e88180a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.039707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6b27-fd98-423b-95dc-454e88180a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.045891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09736847-eabc-44fd-902c-e4cef4841f5b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.051033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc8c2c-7eaa-4cf3-8622-eee34cadbaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.051225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac6b27-fd98-423b-95dc-454e88180a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.051681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc8c2c-7eaa-4cf3-8622-eee34cadbaf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.052063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc8c2c-7eaa-4cf3-8622-eee34cadbaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.052343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc8c2c-7eaa-4cf3-8622-eee34cadbaf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.052459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac6b27-fd98-423b-95dc-454e88180a3c service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.053160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc8c2c-7eaa-4cf3-8622-eee34cadbaf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.055401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc8c2c-7eaa-4cf3-8622-eee34cadbaf7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.068847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 997/2012] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:08.081468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1016/2013] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:07 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:08.085675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac6b27-fd98-423b-95dc-454e88180a3c service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.097035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac6b27-fd98-423b-95dc-454e88180a3c service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.103461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a59eb-1739-4ee8-a557-34d6dabd6a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.104627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a59eb-1739-4ee8-a557-34d6dabd6a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.104984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a59eb-1739-4ee8-a557-34d6dabd6a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.105299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a59eb-1739-4ee8-a557-34d6dabd6a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.105986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a59eb-1739-4ee8-a557-34d6dabd6a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.108072 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a59eb-1739-4ee8-a557-34d6dabd6a99 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.120959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 998/2014] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:08.125967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc8c2c-7eaa-4cf3-8622-eee34cadbaf7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.140983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc8c2c-7eaa-4cf3-8622-eee34cadbaf7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.146679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82886cee-7a37-4386-b5b1-d57629f78905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.147667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82886cee-7a37-4386-b5b1-d57629f78905 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.147667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82886cee-7a37-4386-b5b1-d57629f78905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.147883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82886cee-7a37-4386-b5b1-d57629f78905 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.148525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82886cee-7a37-4386-b5b1-d57629f78905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.150132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82886cee-7a37-4386-b5b1-d57629f78905 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.168811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1017/2015] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:08.171844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a59eb-1739-4ee8-a557-34d6dabd6a99 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.182902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a59eb-1739-4ee8-a557-34d6dabd6a99 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.189036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2dfa6-291f-4411-a6b0-e141543ba526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.189742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2dfa6-291f-4411-a6b0-e141543ba526 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.190014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2dfa6-291f-4411-a6b0-e141543ba526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.190290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2dfa6-291f-4411-a6b0-e141543ba526 admin admin] PATH_INFO: `/v3/projects/d0cc98dfd40c49d0877dc4c2b05457d1/users/38aa2037ced04b379450fd9e73e498d1/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.206100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:08.211922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2dfa6-291f-4411-a6b0-e141543ba526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0cc98dfd40c49d0877dc4c2b05457d1, user_id=38aa2037ced04b379450fd9e73e498d1, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.215803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2dfa6-291f-4411-a6b0-e141543ba526 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.241204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 999/2016] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:08 2022] PUT /identity/v3/projects/d0cc98dfd40c49d0877dc4c2b05457d1/users/38aa2037ced04b379450fd9e73e498d1/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:08.255385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287200c-243b-4473-a606-875973f8c6cd None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.255679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287200c-243b-4473-a606-875973f8c6cd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.257024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287200c-243b-4473-a606-875973f8c6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.301564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e287200c-243b-4473-a606-875973f8c6cd None None] MFA Rules not processed for user `38aa2037ced04b379450fd9e73e498d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:08.302922 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e287200c-243b-4473-a606-875973f8c6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:08.363416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:08.403459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1018/2017] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:08.409157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2dfa6-291f-4411-a6b0-e141543ba526 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.461738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287200c-243b-4473-a606-875973f8c6cd None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.465340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e287200c-243b-4473-a606-875973f8c6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:08.534351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2dfa6-291f-4411-a6b0-e141543ba526 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.546742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896b5c4-4071-44b0-bdc0-df1d605caf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.547706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896b5c4-4071-44b0-bdc0-df1d605caf10 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.548149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896b5c4-4071-44b0-bdc0-df1d605caf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.548680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896b5c4-4071-44b0-bdc0-df1d605caf10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.549714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896b5c4-4071-44b0-bdc0-df1d605caf10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.553160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896b5c4-4071-44b0-bdc0-df1d605caf10 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.610572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287200c-243b-4473-a606-875973f8c6cd None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.611670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1000/2018] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:08 2022] POST /identity/v3/projects => generated 381 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:08.618565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896b5c4-4071-44b0-bdc0-df1d605caf10 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.743538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.744473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.744962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.745447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] PATH_INFO: `/v3/projects/b43c4817974a4d7f85dedbc102c083ea` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.746976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43c4817974a4d7f85dedbc102c083ea)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.750704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.774184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896b5c4-4071-44b0-bdc0-df1d605caf10 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.780698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ccc75-02ba-42ab-95ea-79c64b0f6640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.781480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ccc75-02ba-42ab-95ea-79c64b0f6640 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.781757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ccc75-02ba-42ab-95ea-79c64b0f6640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.782026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ccc75-02ba-42ab-95ea-79c64b0f6640 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.782844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ccc75-02ba-42ab-95ea-79c64b0f6640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.784875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ccc75-02ba-42ab-95ea-79c64b0f6640 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.818920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1001/2019] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:08 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:08.823482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ccc75-02ba-42ab-95ea-79c64b0f6640 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.835735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ccc75-02ba-42ab-95ea-79c64b0f6640 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.846654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43c4817974a4d7f85dedbc102c083ea', 'request_id': 'req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:08.853521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43c4817974a4d7f85dedbc102c083ea', 'request_id': 'req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:08.864447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43c4817974a4d7f85dedbc102c083ea', 'request_id': 'req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:08.880453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] The token cache is being invalidate because project b43c4817974a4d7f85dedbc102c083ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:08.880817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:08.881580 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:08.883413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1019/2020] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:08 2022] DELETE /identity/v3/projects/b43c4817974a4d7f85dedbc102c083ea => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:08.887700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.893815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:08.912546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:08.973595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e74cd0-ba85-4f68-b8b5-25e91c11dc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.974426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e74cd0-ba85-4f68-b8b5-25e91c11dc8e admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.975093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e74cd0-ba85-4f68-b8b5-25e91c11dc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.975274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e74cd0-ba85-4f68-b8b5-25e91c11dc8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.976317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e74cd0-ba85-4f68-b8b5-25e91c11dc8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.979599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e74cd0-ba85-4f68-b8b5-25e91c11dc8e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.991041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:09.002301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1002/2021] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:09.004950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e74cd0-ba85-4f68-b8b5-25e91c11dc8e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:09.016749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f3e74cd0-ba85-4f68-b8b5-25e91c11dc8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:09.024255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2cbcf-1cf2-4d81-ba68-5668fbf36b77 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:09.031328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:09.032089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.032930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.032930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] PATH_INFO: `/v3/users/e01dcd648c044b57824959b59723b4b2` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.033307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e74cd0-ba85-4f68-b8b5-25e91c11dc8e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:09.034156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e01dcd648c044b57824959b59723b4b2)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:09.036481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:09.038501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361488-4d7d-44d9-b1f3-5f7e6a53c344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:09.039166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361488-4d7d-44d9-b1f3-5f7e6a53c344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.039607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361488-4d7d-44d9-b1f3-5f7e6a53c344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.039886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361488-4d7d-44d9-b1f3-5f7e6a53c344 admin admin] PATH_INFO: `/v3/projects/f0e02f57ee8c45ae856e6a81d8481440/users/e80e3a1df93646c981e7183f05ef6d43/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.060202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5361488-4d7d-44d9-b1f3-5f7e6a53c344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e02f57ee8c45ae856e6a81d8481440, user_id=e80e3a1df93646c981e7183f05ef6d43, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:09.063486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5361488-4d7d-44d9-b1f3-5f7e6a53c344 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:09.081127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1003/2022] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:09 2022] PUT /identity/v3/projects/f0e02f57ee8c45ae856e6a81d8481440/users/e80e3a1df93646c981e7183f05ef6d43/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:09.083793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361488-4d7d-44d9-b1f3-5f7e6a53c344 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:09.098040 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e01dcd648c044b57824959b59723b4b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.106847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e01dcd648c044b57824959b59723b4b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.117184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e01dcd648c044b57824959b59723b4b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.118026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e01dcd648c044b57824959b59723b4b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.132821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e01dcd648c044b57824959b59723b4b2'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.140156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1020/2023] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:08 2022] DELETE /identity/v3/users/e01dcd648c044b57824959b59723b4b2 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:09.165027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361488-4d7d-44d9-b1f3-5f7e6a53c344 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:09.236795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ec8b1-ac22-491c-aca0-a1a77aa3c891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:09.237493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ec8b1-ac22-491c-aca0-a1a77aa3c891 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.237770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ec8b1-ac22-491c-aca0-a1a77aa3c891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.238038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ec8b1-ac22-491c-aca0-a1a77aa3c891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.247671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ec8b1-ac22-491c-aca0-a1a77aa3c891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:09.249636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ec8b1-ac22-491c-aca0-a1a77aa3c891 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:09.269742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1004/2024] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:09.274035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:09.275557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:09.292668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be58cab-0621-436c-90d1-2cc420d370fa admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:09.299677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8b8cb-2d4d-4844-a2eb-64982da4c381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:09.300869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8b8cb-2d4d-4844-a2eb-64982da4c381 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.301111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8b8cb-2d4d-4844-a2eb-64982da4c381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.302083 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8b8cb-2d4d-4844-a2eb-64982da4c381 admin admin] PATH_INFO: `/v3/projects/f0e02f57ee8c45ae856e6a81d8481440/users/e80e3a1df93646c981e7183f05ef6d43/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.304226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8b8cb-2d4d-4844-a2eb-64982da4c381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e02f57ee8c45ae856e6a81d8481440, user_id=e80e3a1df93646c981e7183f05ef6d43, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:09.306791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8b8cb-2d4d-4844-a2eb-64982da4c381 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:09.340500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1021/2025] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:09 2022] PUT /identity/v3/projects/f0e02f57ee8c45ae856e6a81d8481440/users/e80e3a1df93646c981e7183f05ef6d43/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:09.351664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0e8c3-2e16-444b-8183-a2cad31c0550 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.351980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0e8c3-2e16-444b-8183-a2cad31c0550 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.352225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0e8c3-2e16-444b-8183-a2cad31c0550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.398409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-13c0e8c3-2e16-444b-8183-a2cad31c0550 None None] MFA Rules not processed for user `e80e3a1df93646c981e7183f05ef6d43`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:09.399696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-13c0e8c3-2e16-444b-8183-a2cad31c0550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:09.487262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8b8cb-2d4d-4844-a2eb-64982da4c381 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:09.489207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1db8b8cb-2d4d-4844-a2eb-64982da4c381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:09.525731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1005/2026] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:09.534557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0e8c3-2e16-444b-8183-a2cad31c0550 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:09.535967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-13c0e8c3-2e16-444b-8183-a2cad31c0550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:09.543138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:09.577189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8b8cb-2d4d-4844-a2eb-64982da4c381 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:09.582304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:09.583036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.583379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.583713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] PATH_INFO: `/v3/projects/2b662abf767c490d899d86cb6cd66bf7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.584912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b662abf767c490d899d86cb6cd66bf7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:09.586977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:09.608948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0e8c3-2e16-444b-8183-a2cad31c0550 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:09.613123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21613c-f2c9-4bc0-a489-d0288b0df8dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:09.613946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21613c-f2c9-4bc0-a489-d0288b0df8dd service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.614723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21613c-f2c9-4bc0-a489-d0288b0df8dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.614723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21613c-f2c9-4bc0-a489-d0288b0df8dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.616695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:09.625744 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21613c-f2c9-4bc0-a489-d0288b0df8dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:09.627240 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21613c-f2c9-4bc0-a489-d0288b0df8dd service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:09.639069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1006/2027] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:09.667358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b662abf767c490d899d86cb6cd66bf7', 'request_id': 'req-0fb7235a-0cf8-4a02-a28c-0959abf107d6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.676557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b662abf767c490d899d86cb6cd66bf7', 'request_id': 'req-0fb7235a-0cf8-4a02-a28c-0959abf107d6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.691046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b662abf767c490d899d86cb6cd66bf7', 'request_id': 'req-0fb7235a-0cf8-4a02-a28c-0959abf107d6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.712490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] The token cache is being invalidate because project 2b662abf767c490d899d86cb6cd66bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:09.712799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.713574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0fb7235a-0cf8-4a02-a28c-0959abf107d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:09.715479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1022/2028] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:09 2022] DELETE /identity/v3/projects/2b662abf767c490d899d86cb6cd66bf7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:09.721249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a5b47-6e63-4dc2-abd7-0c53e6281d7c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.721545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a5b47-6e63-4dc2-abd7-0c53e6281d7c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.721944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a5b47-6e63-4dc2-abd7-0c53e6281d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.752933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3e8a5b47-6e63-4dc2-abd7-0c53e6281d7c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:09.754522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3e8a5b47-6e63-4dc2-abd7-0c53e6281d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:09.847686 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73391996-f3c4-49dd-8ef0-44e2b765f4e3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.848589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73391996-f3c4-49dd-8ef0-44e2b765f4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.849014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73391996-f3c4-49dd-8ef0-44e2b765f4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.857569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1007/2029] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:09.873696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-73391996-f3c4-49dd-8ef0-44e2b765f4e3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:09.876245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-73391996-f3c4-49dd-8ef0-44e2b765f4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:09.888060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1023/2030] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:09.890019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a5b47-6e63-4dc2-abd7-0c53e6281d7c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:09.901086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a5b47-6e63-4dc2-abd7-0c53e6281d7c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:09.906613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0a92-c661-461e-89df-52a5a01d62ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:09.907227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0a92-c661-461e-89df-52a5a01d62ce admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.907512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0a92-c661-461e-89df-52a5a01d62ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.907786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0a92-c661-461e-89df-52a5a01d62ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.908440 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f0a92-c661-461e-89df-52a5a01d62ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:09.908884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f0a92-c661-461e-89df-52a5a01d62ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:09.910548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f0a92-c661-461e-89df-52a5a01d62ce admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:09.925440 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1008/2031] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:09.929992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73391996-f3c4-49dd-8ef0-44e2b765f4e3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:09.941741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73391996-f3c4-49dd-8ef0-44e2b765f4e3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:09.946673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2a9f9-9ab3-4812-a6ae-a6f445432a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:09.947272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2a9f9-9ab3-4812-a6ae-a6f445432a15 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.947552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2a9f9-9ab3-4812-a6ae-a6f445432a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.947879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2a9f9-9ab3-4812-a6ae-a6f445432a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.948580 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2a9f9-9ab3-4812-a6ae-a6f445432a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:09.950535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2a9f9-9ab3-4812-a6ae-a6f445432a15 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:09.986726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1024/2032] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:09 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:09.991893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0a92-c661-461e-89df-52a5a01d62ce admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.006402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0a92-c661-461e-89df-52a5a01d62ce admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.089438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677521db-f9af-4b56-93b6-68c8033a4323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.090059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677521db-f9af-4b56-93b6-68c8033a4323 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.090341 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677521db-f9af-4b56-93b6-68c8033a4323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.091219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677521db-f9af-4b56-93b6-68c8033a4323 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.091219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677521db-f9af-4b56-93b6-68c8033a4323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.093576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677521db-f9af-4b56-93b6-68c8033a4323 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.123134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2a9f9-9ab3-4812-a6ae-a6f445432a15 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.126327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1009/2033] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:09 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:10.129985 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-01e2a9f9-9ab3-4812-a6ae-a6f445432a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:10.130389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677521db-f9af-4b56-93b6-68c8033a4323 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.141904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677521db-f9af-4b56-93b6-68c8033a4323 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.147634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f660e-9875-4279-b1cd-76db6f8b04cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.148248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f660e-9875-4279-b1cd-76db6f8b04cc admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.148533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f660e-9875-4279-b1cd-76db6f8b04cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.148929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f660e-9875-4279-b1cd-76db6f8b04cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.149736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f660e-9875-4279-b1cd-76db6f8b04cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.151380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f660e-9875-4279-b1cd-76db6f8b04cc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.169653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1010/2034] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:10.170591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f660e-9875-4279-b1cd-76db6f8b04cc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.181986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f660e-9875-4279-b1cd-76db6f8b04cc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.189135 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4bf45-a714-4bbe-81e8-96fca21ff15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.189760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4bf45-a714-4bbe-81e8-96fca21ff15a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.190040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4bf45-a714-4bbe-81e8-96fca21ff15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.190335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4bf45-a714-4bbe-81e8-96fca21ff15a admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.190996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4bf45-a714-4bbe-81e8-96fca21ff15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.192579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4bf45-a714-4bbe-81e8-96fca21ff15a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.208859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1011/2035] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:10.209799 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4bf45-a714-4bbe-81e8-96fca21ff15a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.216975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:10.229875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4bf45-a714-4bbe-81e8-96fca21ff15a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.238731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a76021-a617-4c70-9a0a-1594904d52cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.239659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a76021-a617-4c70-9a0a-1594904d52cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.240014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a76021-a617-4c70-9a0a-1594904d52cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.240359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a76021-a617-4c70-9a0a-1594904d52cc admin admin] PATH_INFO: `/v3/projects/af2a4ec9e85a4850a0054ad74a859605/users/db88a8f41bf64cd08bc4ba0f22f268ae/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.271276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a76021-a617-4c70-9a0a-1594904d52cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2a4ec9e85a4850a0054ad74a859605, user_id=db88a8f41bf64cd08bc4ba0f22f268ae, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.272832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2a9f9-9ab3-4812-a6ae-a6f445432a15 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.275727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a76021-a617-4c70-9a0a-1594904d52cc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.279693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb82830-cb8a-433d-a6a0-9180c3837897 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80e3a1df93646c981e7183f05ef6d43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0e02f57ee8c45ae856e6a81d8481440', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.281522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb82830-cb8a-433d-a6a0-9180c3837897 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.281824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb82830-cb8a-433d-a6a0-9180c3837897 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.282171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb82830-cb8a-433d-a6a0-9180c3837897 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.282905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb82830-cb8a-433d-a6a0-9180c3837897 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:10.283287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb82830-cb8a-433d-a6a0-9180c3837897 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.285015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb82830-cb8a-433d-a6a0-9180c3837897 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.315234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1012/2036] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:10 2022] PUT /identity/v3/projects/af2a4ec9e85a4850a0054ad74a859605/users/db88a8f41bf64cd08bc4ba0f22f268ae/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:10.316996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81060d47-ed3f-45c8-b363-83ef2db9ea70 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.317717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1025/2037] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:10.320900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81060d47-ed3f-45c8-b363-83ef2db9ea70 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.321221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81060d47-ed3f-45c8-b363-83ef2db9ea70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.322546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb82830-cb8a-433d-a6a0-9180c3837897 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.344162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb82830-cb8a-433d-a6a0-9180c3837897 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.374409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-81060d47-ed3f-45c8-b363-83ef2db9ea70 None None] MFA Rules not processed for user `db88a8f41bf64cd08bc4ba0f22f268ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:10.378118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-81060d47-ed3f-45c8-b363-83ef2db9ea70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:10.454906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc9b21-1f47-465b-85b1-9821f6a55e98 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80e3a1df93646c981e7183f05ef6d43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0e02f57ee8c45ae856e6a81d8481440', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.455616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc9b21-1f47-465b-85b1-9821f6a55e98 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.456048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc9b21-1f47-465b-85b1-9821f6a55e98 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.456467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc9b21-1f47-465b-85b1-9821f6a55e98 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.457026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc9b21-1f47-465b-85b1-9821f6a55e98 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.460862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc9b21-1f47-465b-85b1-9821f6a55e98 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.494711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1013/2038] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:10.500560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81060d47-ed3f-45c8-b363-83ef2db9ea70 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.517123 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:10.528805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1026/2039] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:10 2022] POST /identity/v3/projects => generated 355 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:10.538666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc9b21-1f47-465b-85b1-9821f6a55e98 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.601774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81060d47-ed3f-45c8-b363-83ef2db9ea70 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.646704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc9b21-1f47-465b-85b1-9821f6a55e98 tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.651952 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18cf67-de55-4b46-bcc3-2c07ec5871be tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80e3a1df93646c981e7183f05ef6d43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0e02f57ee8c45ae856e6a81d8481440', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.652647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18cf67-de55-4b46-bcc3-2c07ec5871be tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.652999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18cf67-de55-4b46-bcc3-2c07ec5871be tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.653550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18cf67-de55-4b46-bcc3-2c07ec5871be tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.656098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18cf67-de55-4b46-bcc3-2c07ec5871be tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:10.656494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18cf67-de55-4b46-bcc3-2c07ec5871be tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.658231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18cf67-de55-4b46-bcc3-2c07ec5871be tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.668300 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e80f3-1aac-4ffd-8c63-476898ff4adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.668912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e80f3-1aac-4ffd-8c63-476898ff4adb admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.669248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e80f3-1aac-4ffd-8c63-476898ff4adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.669510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e80f3-1aac-4ffd-8c63-476898ff4adb admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.670146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e80f3-1aac-4ffd-8c63-476898ff4adb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.672836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e80f3-1aac-4ffd-8c63-476898ff4adb admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.674194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1027/2040] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:10.707144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:10.722653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1014/2041] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:10 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:10.722881 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18cf67-de55-4b46-bcc3-2c07ec5871be tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.724557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e80f3-1aac-4ffd-8c63-476898ff4adb admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.736324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18cf67-de55-4b46-bcc3-2c07ec5871be tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.818382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80e3a1df93646c981e7183f05ef6d43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0e02f57ee8c45ae856e6a81d8481440', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.819173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.819292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.819562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] PATH_INFO: `/v3/projects/3732f81093f14b338c2081a3569707b3` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.820741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3732f81093f14b338c2081a3569707b3)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.822719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.823919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e80f3-1aac-4ffd-8c63-476898ff4adb admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.829442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34854bc-86f6-4dca-963f-77f34291b662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.830113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34854bc-86f6-4dca-963f-77f34291b662 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.830400 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34854bc-86f6-4dca-963f-77f34291b662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.830679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34854bc-86f6-4dca-963f-77f34291b662 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.831313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34854bc-86f6-4dca-963f-77f34291b662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.833249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34854bc-86f6-4dca-963f-77f34291b662 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.857727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1015/2042] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:10 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:10.859152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34854bc-86f6-4dca-963f-77f34291b662 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.869250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34854bc-86f6-4dca-963f-77f34291b662 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.870723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3732f81093f14b338c2081a3569707b3', 'request_id': 'req-21fd30de-2633-4733-8f89-e59bcbfd41ec'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:10.878061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3732f81093f14b338c2081a3569707b3', 'request_id': 'req-21fd30de-2633-4733-8f89-e59bcbfd41ec'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:10.886099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3732f81093f14b338c2081a3569707b3', 'request_id': 'req-21fd30de-2633-4733-8f89-e59bcbfd41ec'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:10.899912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] The token cache is being invalidate because project 3732f81093f14b338c2081a3569707b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:10.900317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:10.900975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:10.902524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1028/2043] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:10 2022] DELETE /identity/v3/projects/3732f81093f14b338c2081a3569707b3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:10.935633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cdd07-eb18-4a04-921c-1add3cbaf7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.936799 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cdd07-eb18-4a04-921c-1add3cbaf7ef admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.937110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cdd07-eb18-4a04-921c-1add3cbaf7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.937415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cdd07-eb18-4a04-921c-1add3cbaf7ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.938065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cdd07-eb18-4a04-921c-1add3cbaf7ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:10.939666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cdd07-eb18-4a04-921c-1add3cbaf7ef admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:10.953638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1016/2044] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:10.954827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.956334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:10.972768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd30de-2633-4733-8f89-e59bcbfd41ec tempest-VolumeQuotaClassesTest-1860618869 tempest-VolumeQuotaClassesTest-1860618869-project-admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.978173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab951009-38a1-4b92-b8b9-4cc8f1f403a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.978836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab951009-38a1-4b92-b8b9-4cc8f1f403a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.979120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab951009-38a1-4b92-b8b9-4cc8f1f403a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.979397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab951009-38a1-4b92-b8b9-4cc8f1f403a9 admin admin] PATH_INFO: `/v3/projects/a68bc870a3a44f4cb2850b588c43e353/users/322db9c2adb243f0b346346e23acbde7/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.999322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab951009-38a1-4b92-b8b9-4cc8f1f403a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68bc870a3a44f4cb2850b588c43e353, user_id=322db9c2adb243f0b346346e23acbde7, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.002953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab951009-38a1-4b92-b8b9-4cc8f1f403a9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.013561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.021888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1029/2045] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:10 2022] PUT /identity/v3/projects/a68bc870a3a44f4cb2850b588c43e353/users/322db9c2adb243f0b346346e23acbde7/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:11.026114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cdd07-eb18-4a04-921c-1add3cbaf7ef admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.037046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cdd07-eb18-4a04-921c-1add3cbaf7ef admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.119991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f222915-f987-48c2-8a04-85c6186b21cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.120611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222915-f987-48c2-8a04-85c6186b21cd admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.120944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222915-f987-48c2-8a04-85c6186b21cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.121412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222915-f987-48c2-8a04-85c6186b21cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.122080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f222915-f987-48c2-8a04-85c6186b21cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.125019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f222915-f987-48c2-8a04-85c6186b21cd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.132139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab951009-38a1-4b92-b8b9-4cc8f1f403a9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.134080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ab951009-38a1-4b92-b8b9-4cc8f1f403a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:11.141117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1017/2046] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:11.143568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f222915-f987-48c2-8a04-85c6186b21cd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.149840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab951009-38a1-4b92-b8b9-4cc8f1f403a9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.153621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f222915-f987-48c2-8a04-85c6186b21cd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.154606 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.155410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.155643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.155914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] PATH_INFO: `/v3/users/38aa2037ced04b379450fd9e73e498d1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.157136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] RBAC: Authorizing `identity:delete_user(user_id=38aa2037ced04b379450fd9e73e498d1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.158290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e54e84-ace4-43e4-a6ae-4a59e4aa8f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.158932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e54e84-ace4-43e4-a6ae-4a59e4aa8f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.159044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.159207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e54e84-ace4-43e4-a6ae-4a59e4aa8f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.159498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e54e84-ace4-43e4-a6ae-4a59e4aa8f66 admin admin] PATH_INFO: `/v3/projects/a68bc870a3a44f4cb2850b588c43e353/users/322db9c2adb243f0b346346e23acbde7/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.161407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e54e84-ace4-43e4-a6ae-4a59e4aa8f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68bc870a3a44f4cb2850b588c43e353, user_id=322db9c2adb243f0b346346e23acbde7, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.164008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e54e84-ace4-43e4-a6ae-4a59e4aa8f66 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.178203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1018/2047] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:11 2022] PUT /identity/v3/projects/a68bc870a3a44f4cb2850b588c43e353/users/322db9c2adb243f0b346346e23acbde7/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:11.189837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91589730-fe1e-43a1-97e6-df9de0b61408 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.190127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91589730-fe1e-43a1-97e6-df9de0b61408 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.190379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91589730-fe1e-43a1-97e6-df9de0b61408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.209619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38aa2037ced04b379450fd9e73e498d1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.210386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.217052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38aa2037ced04b379450fd9e73e498d1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.221274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-91589730-fe1e-43a1-97e6-df9de0b61408 None None] MFA Rules not processed for user `322db9c2adb243f0b346346e23acbde7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:11.222625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-91589730-fe1e-43a1-97e6-df9de0b61408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:11.224714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38aa2037ced04b379450fd9e73e498d1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.225362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38aa2037ced04b379450fd9e73e498d1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.241934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-ba858e80-a585-4a41-b08c-5e97a73fa42e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38aa2037ced04b379450fd9e73e498d1'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.248294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1030/2048] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:11 2022] DELETE /identity/v3/users/38aa2037ced04b379450fd9e73e498d1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:11.253325 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd1cbe-f610-40eb-9534-4fa23f8453e3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.253616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd1cbe-f610-40eb-9534-4fa23f8453e3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.253889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd1cbe-f610-40eb-9534-4fa23f8453e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.275245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-74fd1cbe-f610-40eb-9534-4fa23f8453e3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:11.276564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-74fd1cbe-f610-40eb-9534-4fa23f8453e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:11.300743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1019/2049] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:11.312495 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91589730-fe1e-43a1-97e6-df9de0b61408 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.314031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-91589730-fe1e-43a1-97e6-df9de0b61408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:11.350523 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1031/2050] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:11.361831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd1cbe-f610-40eb-9534-4fa23f8453e3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.363455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-74fd1cbe-f610-40eb-9534-4fa23f8453e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:11.383721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91589730-fe1e-43a1-97e6-df9de0b61408 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.430683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd1cbe-f610-40eb-9534-4fa23f8453e3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.434822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141464b-01a1-4e6b-8ab0-08b35b7b66ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.435286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141464b-01a1-4e6b-8ab0-08b35b7b66ea service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.435571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141464b-01a1-4e6b-8ab0-08b35b7b66ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.435841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141464b-01a1-4e6b-8ab0-08b35b7b66ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.443450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e5136-8513-46c7-9aad-e0403000c27d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.444178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e5136-8513-46c7-9aad-e0403000c27d service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.444651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e5136-8513-46c7-9aad-e0403000c27d service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.444991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e5136-8513-46c7-9aad-e0403000c27d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.446872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141464b-01a1-4e6b-8ab0-08b35b7b66ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.448294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141464b-01a1-4e6b-8ab0-08b35b7b66ea service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.459151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1032/2051] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:11.524327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e5136-8513-46c7-9aad-e0403000c27d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.527769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e5136-8513-46c7-9aad-e0403000c27d service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.534617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.543525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1020/2052] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:11.811603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141464b-01a1-4e6b-8ab0-08b35b7b66ea service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.813421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e141464b-01a1-4e6b-8ab0-08b35b7b66ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:11.840799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141464b-01a1-4e6b-8ab0-08b35b7b66ea service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.856134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.856784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.857062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.857381 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] PATH_INFO: `/v3/projects/d0cc98dfd40c49d0877dc4c2b05457d1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.859752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0cc98dfd40c49d0877dc4c2b05457d1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.873628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.873628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.923518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0cc98dfd40c49d0877dc4c2b05457d1', 'request_id': 'req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.930371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0cc98dfd40c49d0877dc4c2b05457d1', 'request_id': 'req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.945111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0cc98dfd40c49d0877dc4c2b05457d1', 'request_id': 'req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.958075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] The token cache is being invalidate because project d0cc98dfd40c49d0877dc4c2b05457d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:11.958639 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.960021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:11.967003 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1033/2053] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:11 2022] DELETE /identity/v3/projects/d0cc98dfd40c49d0877dc4c2b05457d1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:11.967091 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e5136-8513-46c7-9aad-e0403000c27d service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.976977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e5136-8513-46c7-9aad-e0403000c27d service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.052055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.052692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.052972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.053274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] PATH_INFO: `/v3/users/e80e3a1df93646c981e7183f05ef6d43` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.054419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e80e3a1df93646c981e7183f05ef6d43)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:12.056357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:12.109561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e80e3a1df93646c981e7183f05ef6d43'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.118381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e80e3a1df93646c981e7183f05ef6d43'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.119338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e80e3a1df93646c981e7183f05ef6d43'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.120005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e80e3a1df93646c981e7183f05ef6d43'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.134016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7e7a1fc5-1a83-4239-958a-b48aa93ce328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e80e3a1df93646c981e7183f05ef6d43'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.147442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1021/2054] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:11 2022] DELETE /identity/v3/users/e80e3a1df93646c981e7183f05ef6d43 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:12.351818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.353313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:12.375004 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7795d35-c42c-4f7f-8528-4ee4e2211dd6 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.470665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.471364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.471646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.471914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] PATH_INFO: `/v3/users/db88a8f41bf64cd08bc4ba0f22f268ae` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.473154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db88a8f41bf64cd08bc4ba0f22f268ae)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:12.475148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:12.550516 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db88a8f41bf64cd08bc4ba0f22f268ae'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.567164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db88a8f41bf64cd08bc4ba0f22f268ae'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.579190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db88a8f41bf64cd08bc4ba0f22f268ae'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.580218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db88a8f41bf64cd08bc4ba0f22f268ae'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.602402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db88a8f41bf64cd08bc4ba0f22f268ae'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:12.616427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1034/2055] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:12 2022] DELETE /identity/v3/users/db88a8f41bf64cd08bc4ba0f22f268ae => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:12.620756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf849e6-0311-450d-ad82-7e1e3865002b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.623324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf849e6-0311-450d-ad82-7e1e3865002b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.623676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf849e6-0311-450d-ad82-7e1e3865002b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.666995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2cf849e6-0311-450d-ad82-7e1e3865002b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:12.668379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2cf849e6-0311-450d-ad82-7e1e3865002b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:12.783310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1022/2056] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:12.794907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.800711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:12.842104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:12.890650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1ac52-a7d3-4789-8ba4-5e4af655de41 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.895814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11757e-60bf-43fc-a676-b76986adc2a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.895814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11757e-60bf-43fc-a676-b76986adc2a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.896070 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11757e-60bf-43fc-a676-b76986adc2a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.896187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11757e-60bf-43fc-a676-b76986adc2a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.907474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11757e-60bf-43fc-a676-b76986adc2a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:12.909065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11757e-60bf-43fc-a676-b76986adc2a3 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:12.923836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1035/2057] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:13.070683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf849e6-0311-450d-ad82-7e1e3865002b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.072427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2cf849e6-0311-450d-ad82-7e1e3865002b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:13.089616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf849e6-0311-450d-ad82-7e1e3865002b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.100733 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.102369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.102726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.103410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] PATH_INFO: `/v3/projects/f0e02f57ee8c45ae856e6a81d8481440` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.104675 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e02f57ee8c45ae856e6a81d8481440)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.107335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.151022 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:13.200628 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e02f57ee8c45ae856e6a81d8481440', 'request_id': 'req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.217216 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e02f57ee8c45ae856e6a81d8481440', 'request_id': 'req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.234167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e02f57ee8c45ae856e6a81d8481440', 'request_id': 'req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.265385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] The token cache is being invalidate because project f0e02f57ee8c45ae856e6a81d8481440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:13.265689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.266437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.268334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1023/2058] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:13 2022] DELETE /identity/v3/projects/f0e02f57ee8c45ae856e6a81d8481440 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:13.273352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2aee7-c427-4668-80de-87f6a02f93c1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.273661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2aee7-c427-4668-80de-87f6a02f93c1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.273865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2aee7-c427-4668-80de-87f6a02f93c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.281017 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.282741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:13.318490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-77a2aee7-c427-4668-80de-87f6a02f93c1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:13.319794 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-77a2aee7-c427-4668-80de-87f6a02f93c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:13.398914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9e2b8-9f94-4f17-ae45-d3e8dfedbd73 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.405241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1036/2059] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:13.405809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.414732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831aa99-ad7f-457f-b303-98f57c74064c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.415087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831aa99-ad7f-457f-b303-98f57c74064c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.415439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831aa99-ad7f-457f-b303-98f57c74064c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.418254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.418847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.419507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] PATH_INFO: `/v3/projects/af2a4ec9e85a4850a0054ad74a859605` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.422006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2a4ec9e85a4850a0054ad74a859605)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.424331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.444136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c831aa99-ad7f-457f-b303-98f57c74064c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:13.445579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c831aa99-ad7f-457f-b303-98f57c74064c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:13.456742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1037/2060] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:13.462256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831aa99-ad7f-457f-b303-98f57c74064c None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.476668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831aa99-ad7f-457f-b303-98f57c74064c None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.487466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2a4ec9e85a4850a0054ad74a859605', 'request_id': 'req-f358298e-23ff-4c97-8cdc-e59456e61c82'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.495365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2a4ec9e85a4850a0054ad74a859605', 'request_id': 'req-f358298e-23ff-4c97-8cdc-e59456e61c82'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.507059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2a4ec9e85a4850a0054ad74a859605', 'request_id': 'req-f358298e-23ff-4c97-8cdc-e59456e61c82'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.520160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] The token cache is being invalidate because project af2a4ec9e85a4850a0054ad74a859605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:13.520570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.521687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.523297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1024/2061] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:13 2022] DELETE /identity/v3/projects/af2a4ec9e85a4850a0054ad74a859605 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:13.527130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.528608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:13.551205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.551805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.552091 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.552367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.553030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:13.553441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.555078 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.574031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1038/2062] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:13.578748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.581352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:13.585682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:13.600741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f13a3-3180-4958-8fc5-e0e279b787ce admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.605914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041266d-06fc-4510-b8e7-5d25c378c31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.606739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041266d-06fc-4510-b8e7-5d25c378c31b admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.606891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041266d-06fc-4510-b8e7-5d25c378c31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.607155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041266d-06fc-4510-b8e7-5d25c378c31b admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.607906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041266d-06fc-4510-b8e7-5d25c378c31b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.610221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041266d-06fc-4510-b8e7-5d25c378c31b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.623313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358298e-23ff-4c97-8cdc-e59456e61c82 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.634773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.635627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.635909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.636185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] PATH_INFO: `/v3/users/322db9c2adb243f0b346346e23acbde7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.637512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=322db9c2adb243f0b346346e23acbde7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.640679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.653552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1039/2063] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:13 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:13.655531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041266d-06fc-4510-b8e7-5d25c378c31b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.690586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:13.691563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322db9c2adb243f0b346346e23acbde7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.701585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322db9c2adb243f0b346346e23acbde7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.702324 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322db9c2adb243f0b346346e23acbde7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.702953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322db9c2adb243f0b346346e23acbde7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.715428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322db9c2adb243f0b346346e23acbde7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:13.730042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041266d-06fc-4510-b8e7-5d25c378c31b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.734262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1025/2064] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:13 2022] DELETE /identity/v3/users/322db9c2adb243f0b346346e23acbde7 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:13.738979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218abbd-6df0-48f3-8643-15b354e1a3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.739733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218abbd-6df0-48f3-8643-15b354e1a3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.740070 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218abbd-6df0-48f3-8643-15b354e1a3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.740534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218abbd-6df0-48f3-8643-15b354e1a3d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.741452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218abbd-6df0-48f3-8643-15b354e1a3d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.743827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218abbd-6df0-48f3-8643-15b354e1a3d7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.773180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1040/2065] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:13 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:13.774355 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.775899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:13.847968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff73fc3-3afc-4363-aa0f-cf00699d8643 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.854048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f59f9c-ae0e-46f8-bda5-9ce727760e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.854702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59f9c-ae0e-46f8-bda5-9ce727760e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.854981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59f9c-ae0e-46f8-bda5-9ce727760e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.855252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59f9c-ae0e-46f8-bda5-9ce727760e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.855905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f59f9c-ae0e-46f8-bda5-9ce727760e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.857544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f59f9c-ae0e-46f8-bda5-9ce727760e46 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.871771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1026/2066] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:13.875955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218abbd-6df0-48f3-8643-15b354e1a3d7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.886212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218abbd-6df0-48f3-8643-15b354e1a3d7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.891048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214978e9-61b7-41bd-9e9b-4c5886ee68a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.891643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214978e9-61b7-41bd-9e9b-4c5886ee68a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.891928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214978e9-61b7-41bd-9e9b-4c5886ee68a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.892195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214978e9-61b7-41bd-9e9b-4c5886ee68a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.893349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214978e9-61b7-41bd-9e9b-4c5886ee68a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.894473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214978e9-61b7-41bd-9e9b-4c5886ee68a8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.905975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1041/2067] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:13.909874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f59f9c-ae0e-46f8-bda5-9ce727760e46 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.919856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f59f9c-ae0e-46f8-bda5-9ce727760e46 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.925906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321b123-89c0-4c30-8377-a8fae576fb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.926605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321b123-89c0-4c30-8377-a8fae576fb31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.926886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321b123-89c0-4c30-8377-a8fae576fb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.927162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321b123-89c0-4c30-8377-a8fae576fb31 admin admin] PATH_INFO: `/v3/projects/5c5256840b8e44d08d798e51886828c6/users/7fe693c86d764e35afb607aed0f75385/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.942742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321b123-89c0-4c30-8377-a8fae576fb31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5256840b8e44d08d798e51886828c6, user_id=7fe693c86d764e35afb607aed0f75385, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.945452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321b123-89c0-4c30-8377-a8fae576fb31 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.960236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1027/2068] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:13 2022] PUT /identity/v3/projects/5c5256840b8e44d08d798e51886828c6/users/7fe693c86d764e35afb607aed0f75385/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:13.973138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c996d-bf85-4786-bd19-b9adf84bd7a0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.973425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c996d-bf85-4786-bd19-b9adf84bd7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.973694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c996d-bf85-4786-bd19-b9adf84bd7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.006377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-321c996d-bf85-4786-bd19-b9adf84bd7a0 None None] MFA Rules not processed for user `7fe693c86d764e35afb607aed0f75385`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:14.007854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-321c996d-bf85-4786-bd19-b9adf84bd7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.027136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321b123-89c0-4c30-8377-a8fae576fb31 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.029571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6321b123-89c0-4c30-8377-a8fae576fb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.065506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:14.101614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1042/2069] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:14.106960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c996d-bf85-4786-bd19-b9adf84bd7a0 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.115049 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321b123-89c0-4c30-8377-a8fae576fb31 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.118455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c996d-bf85-4786-bd19-b9adf84bd7a0 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.119944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.120584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.120939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.121184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] PATH_INFO: `/v3/projects/a68bc870a3a44f4cb2850b588c43e353` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.122397 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68bc870a3a44f4cb2850b588c43e353)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.124495 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.126548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913ab00-3d9b-4add-8a31-978d0fd584c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.127163 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913ab00-3d9b-4add-8a31-978d0fd584c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.127446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913ab00-3d9b-4add-8a31-978d0fd584c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.127708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913ab00-3d9b-4add-8a31-978d0fd584c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.128347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913ab00-3d9b-4add-8a31-978d0fd584c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.130268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913ab00-3d9b-4add-8a31-978d0fd584c5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.164135 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1043/2070] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:14.165101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913ab00-3d9b-4add-8a31-978d0fd584c5 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.182127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68bc870a3a44f4cb2850b588c43e353', 'request_id': 'req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.189023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68bc870a3a44f4cb2850b588c43e353', 'request_id': 'req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.200653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68bc870a3a44f4cb2850b588c43e353', 'request_id': 'req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.218617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] The token cache is being invalidate because project a68bc870a3a44f4cb2850b588c43e353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:14.219085 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.219823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6f528b4b-d7f2-49f2-aa9a-e751a1afcfed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.221838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1028/2071] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:14 2022] DELETE /identity/v3/projects/a68bc870a3a44f4cb2850b588c43e353 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:14.227279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73414a-6b9b-4a8c-be61-fc598580d4cc None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.228088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73414a-6b9b-4a8c-be61-fc598580d4cc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.228504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73414a-6b9b-4a8c-be61-fc598580d4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.247970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913ab00-3d9b-4add-8a31-978d0fd584c5 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.266818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2e73414a-6b9b-4a8c-be61-fc598580d4cc None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:14.268252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2e73414a-6b9b-4a8c-be61-fc598580d4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.270162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:14.330727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec057d-f0c0-410e-aef7-27f0125219a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.331368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec057d-f0c0-410e-aef7-27f0125219a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.331790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec057d-f0c0-410e-aef7-27f0125219a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.332055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec057d-f0c0-410e-aef7-27f0125219a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.332689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec057d-f0c0-410e-aef7-27f0125219a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.335242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec057d-f0c0-410e-aef7-27f0125219a7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.365462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1029/2072] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:14.365666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1044/2073] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:14.368009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917905-dc6a-47e1-a4f9-837573cbb2dd None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.368323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec057d-f0c0-410e-aef7-27f0125219a7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.368570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917905-dc6a-47e1-a4f9-837573cbb2dd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.368854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917905-dc6a-47e1-a4f9-837573cbb2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.370837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d8ec057d-f0c0-410e-aef7-27f0125219a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.390944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec057d-f0c0-410e-aef7-27f0125219a7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.392543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-13917905-dc6a-47e1-a4f9-837573cbb2dd None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:14.393789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-13917905-dc6a-47e1-a4f9-837573cbb2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.396935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fff62-68e6-4795-9853-b58dd7b85069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.398052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fff62-68e6-4795-9853-b58dd7b85069 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.398336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fff62-68e6-4795-9853-b58dd7b85069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.398601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fff62-68e6-4795-9853-b58dd7b85069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.399247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fff62-68e6-4795-9853-b58dd7b85069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.402030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fff62-68e6-4795-9853-b58dd7b85069 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.406572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1030/2074] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:14.407469 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13917905-dc6a-47e1-a4f9-837573cbb2dd None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.417211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1045/2075] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:14.423297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fff62-68e6-4795-9853-b58dd7b85069 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.424092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13917905-dc6a-47e1-a4f9-837573cbb2dd None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.430966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4ea49-1ad7-477a-ae0d-073a6344fce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.431577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4ea49-1ad7-477a-ae0d-073a6344fce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.431851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4ea49-1ad7-477a-ae0d-073a6344fce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.432115 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4ea49-1ad7-477a-ae0d-073a6344fce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.432761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4ea49-1ad7-477a-ae0d-073a6344fce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:14.433057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4ea49-1ad7-477a-ae0d-073a6344fce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.434677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4ea49-1ad7-477a-ae0d-073a6344fce1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.435661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fff62-68e6-4795-9853-b58dd7b85069 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.442441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef853-9b99-4966-84ec-eea0e67dda52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.443232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef853-9b99-4966-84ec-eea0e67dda52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.443614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef853-9b99-4966-84ec-eea0e67dda52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.443989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef853-9b99-4966-84ec-eea0e67dda52 admin admin] PATH_INFO: `/v3/projects/ed3ff784822a4fd79348770f3d290e84/users/5632ed8ec966440380472e659071eb18/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.449943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1031/2076] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:14.454893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4ea49-1ad7-477a-ae0d-073a6344fce1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.468534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4ea49-1ad7-477a-ae0d-073a6344fce1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.469035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef853-9b99-4966-84ec-eea0e67dda52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed3ff784822a4fd79348770f3d290e84, user_id=5632ed8ec966440380472e659071eb18, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.471758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef853-9b99-4966-84ec-eea0e67dda52 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.474097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f242c-3d72-4b0b-967f-41f17875cd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.475049 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f242c-3d72-4b0b-967f-41f17875cd56 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.475594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f242c-3d72-4b0b-967f-41f17875cd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.475804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f242c-3d72-4b0b-967f-41f17875cd56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.476713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f242c-3d72-4b0b-967f-41f17875cd56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.479089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f242c-3d72-4b0b-967f-41f17875cd56 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.491149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1046/2077] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:14 2022] PUT /identity/v3/projects/ed3ff784822a4fd79348770f3d290e84/users/5632ed8ec966440380472e659071eb18/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:14.503033 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef853-9b99-4966-84ec-eea0e67dda52 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.525123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1032/2078] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:14.528363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f242c-3d72-4b0b-967f-41f17875cd56 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.542948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:14.570737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef853-9b99-4966-84ec-eea0e67dda52 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.599777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f242c-3d72-4b0b-967f-41f17875cd56 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.604429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e74972-8318-414a-8231-ed0af08c54fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.605029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e74972-8318-414a-8231-ed0af08c54fe admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.605343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e74972-8318-414a-8231-ed0af08c54fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.605617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e74972-8318-414a-8231-ed0af08c54fe admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.606252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e74972-8318-414a-8231-ed0af08c54fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.608304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e74972-8318-414a-8231-ed0af08c54fe admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.629573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141366b-e3d2-4c63-878b-956046bb0335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.630189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141366b-e3d2-4c63-878b-956046bb0335 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.630483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141366b-e3d2-4c63-878b-956046bb0335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.630738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141366b-e3d2-4c63-878b-956046bb0335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.631417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141366b-e3d2-4c63-878b-956046bb0335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.635736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1033/2079] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:14.636186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141366b-e3d2-4c63-878b-956046bb0335 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.636748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e74972-8318-414a-8231-ed0af08c54fe admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.644687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1047/2080] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:14.648166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141366b-e3d2-4c63-878b-956046bb0335 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.650819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e74972-8318-414a-8231-ed0af08c54fe admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.656241 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3b28a-f1fd-4ab1-b681-06f061fd29c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.656978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3b28a-f1fd-4ab1-b681-06f061fd29c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.657676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3b28a-f1fd-4ab1-b681-06f061fd29c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.658021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3b28a-f1fd-4ab1-b681-06f061fd29c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.658893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3b28a-f1fd-4ab1-b681-06f061fd29c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.659801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141366b-e3d2-4c63-878b-956046bb0335 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.660693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3b28a-f1fd-4ab1-b681-06f061fd29c9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.664767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac824d7-330b-4674-b2a4-bcb0134c86fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.665514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac824d7-330b-4674-b2a4-bcb0134c86fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.665856 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac824d7-330b-4674-b2a4-bcb0134c86fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.666190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac824d7-330b-4674-b2a4-bcb0134c86fc admin admin] PATH_INFO: `/v3/projects/ed3ff784822a4fd79348770f3d290e84/users/5632ed8ec966440380472e659071eb18/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.668324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac824d7-330b-4674-b2a4-bcb0134c86fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed3ff784822a4fd79348770f3d290e84, user_id=5632ed8ec966440380472e659071eb18, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.671477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac824d7-330b-4674-b2a4-bcb0134c86fc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.673533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1034/2081] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:14.678533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3b28a-f1fd-4ab1-b681-06f061fd29c9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.690485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3b28a-f1fd-4ab1-b681-06f061fd29c9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.693164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1048/2082] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:14 2022] PUT /identity/v3/projects/ed3ff784822a4fd79348770f3d290e84/users/5632ed8ec966440380472e659071eb18/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:14.703634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d16c4-1aa4-46e7-abb4-eca7b1dcab76 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.703891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d16c4-1aa4-46e7-abb4-eca7b1dcab76 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.704169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d16c4-1aa4-46e7-abb4-eca7b1dcab76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.715095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:14.746210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-504d16c4-1aa4-46e7-abb4-eca7b1dcab76 None None] MFA Rules not processed for user `5632ed8ec966440380472e659071eb18`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:14.747465 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-504d16c4-1aa4-46e7-abb4-eca7b1dcab76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.758330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b6e85-e5e6-4077-b252-5059a61ddd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.758943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6e85-e5e6-4077-b252-5059a61ddd7a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.759223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6e85-e5e6-4077-b252-5059a61ddd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.759493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6e85-e5e6-4077-b252-5059a61ddd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.761059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b6e85-e5e6-4077-b252-5059a61ddd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.763033 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b6e85-e5e6-4077-b252-5059a61ddd7a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.777984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1035/2083] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:14.778990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b6e85-e5e6-4077-b252-5059a61ddd7a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.790212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b6e85-e5e6-4077-b252-5059a61ddd7a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.796818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69042753-3c9f-45b2-94cd-55534a418664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.797505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69042753-3c9f-45b2-94cd-55534a418664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.797794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69042753-3c9f-45b2-94cd-55534a418664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.798062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69042753-3c9f-45b2-94cd-55534a418664 admin admin] PATH_INFO: `/v3/projects/4fa5a4384ef740f4861c8ff5ee9f6456/users/c176fdd845fd4ffba8ef852ed458d067/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.816698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69042753-3c9f-45b2-94cd-55534a418664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa5a4384ef740f4861c8ff5ee9f6456, user_id=c176fdd845fd4ffba8ef852ed458d067, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.819427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69042753-3c9f-45b2-94cd-55534a418664 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.837107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1036/2084] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:14 2022] PUT /identity/v3/projects/4fa5a4384ef740f4861c8ff5ee9f6456/users/c176fdd845fd4ffba8ef852ed458d067/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:14.840960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1049/2085] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:14.847670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf8131-06a5-4fdc-bd29-8503e64494e9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.848038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf8131-06a5-4fdc-bd29-8503e64494e9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.848367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf8131-06a5-4fdc-bd29-8503e64494e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.852452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d16c4-1aa4-46e7-abb4-eca7b1dcab76 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.853893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-504d16c4-1aa4-46e7-abb4-eca7b1dcab76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.879090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cdcf8131-06a5-4fdc-bd29-8503e64494e9 None None] MFA Rules not processed for user `c176fdd845fd4ffba8ef852ed458d067`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:14.880562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cdcf8131-06a5-4fdc-bd29-8503e64494e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.918784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d16c4-1aa4-46e7-abb4-eca7b1dcab76 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.943052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:14.957368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1037/2086] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:14.970212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf8131-06a5-4fdc-bd29-8503e64494e9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.971746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cdcf8131-06a5-4fdc-bd29-8503e64494e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.977441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4a1c6-5538-4ae2-83ea-15d7ce5c7d87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.978047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4a1c6-5538-4ae2-83ea-15d7ce5c7d87 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.978321 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4a1c6-5538-4ae2-83ea-15d7ce5c7d87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.978585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4a1c6-5538-4ae2-83ea-15d7ce5c7d87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.036108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf8131-06a5-4fdc-bd29-8503e64494e9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.038107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:15.040330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785ab52-cf3a-47e1-b5ae-2f4dab3a435c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:15.041025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785ab52-cf3a-47e1-b5ae-2f4dab3a435c service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.041257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785ab52-cf3a-47e1-b5ae-2f4dab3a435c service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.041525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785ab52-cf3a-47e1-b5ae-2f4dab3a435c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.042857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4a1c6-5538-4ae2-83ea-15d7ce5c7d87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.044666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4a1c6-5538-4ae2-83ea-15d7ce5c7d87 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.050886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785ab52-cf3a-47e1-b5ae-2f4dab3a435c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.052309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785ab52-cf3a-47e1-b5ae-2f4dab3a435c service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.057652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1050/2087] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:15.061264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1038/2088] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:15.089207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4a1c6-5538-4ae2-83ea-15d7ce5c7d87 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:15.171180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4a1c6-5538-4ae2-83ea-15d7ce5c7d87 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.175946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:15.176574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.176856 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.177134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] PATH_INFO: `/v3/users/c176fdd845fd4ffba8ef852ed458d067` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.178302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c176fdd845fd4ffba8ef852ed458d067)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.180295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.230817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c176fdd845fd4ffba8ef852ed458d067'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.240983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c176fdd845fd4ffba8ef852ed458d067'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.252331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c176fdd845fd4ffba8ef852ed458d067'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.253059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c176fdd845fd4ffba8ef852ed458d067'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.268566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c176fdd845fd4ffba8ef852ed458d067'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.278245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1051/2089] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:15 2022] DELETE /identity/v3/users/c176fdd845fd4ffba8ef852ed458d067 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:15.285759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c177a30-2ece-4184-9e93-1e84a5fd00bd None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.286149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c177a30-2ece-4184-9e93-1e84a5fd00bd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.286626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c177a30-2ece-4184-9e93-1e84a5fd00bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.306359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0c177a30-2ece-4184-9e93-1e84a5fd00bd None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:15.307574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0c177a30-2ece-4184-9e93-1e84a5fd00bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:15.381115 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1039/2090] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:15.389117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:15.390647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:15.457416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52065913-f512-4c1b-80cd-082a6d889040 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.461256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58589d76-17a5-4593-a1ff-e14cc567ceee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:15.461865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58589d76-17a5-4593-a1ff-e14cc567ceee service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.462141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58589d76-17a5-4593-a1ff-e14cc567ceee service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.462410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58589d76-17a5-4593-a1ff-e14cc567ceee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.464568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c177a30-2ece-4184-9e93-1e84a5fd00bd None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:15.465969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0c177a30-2ece-4184-9e93-1e84a5fd00bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:15.477373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58589d76-17a5-4593-a1ff-e14cc567ceee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.478827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58589d76-17a5-4593-a1ff-e14cc567ceee service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.489817 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c177a30-2ece-4184-9e93-1e84a5fd00bd None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.491811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1052/2091] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:15.494518 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:15.495142 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.495423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.495693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] PATH_INFO: `/v3/users/7fe693c86d764e35afb607aed0f75385` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.496944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fe693c86d764e35afb607aed0f75385)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.499838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.574094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fe693c86d764e35afb607aed0f75385'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.587572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fe693c86d764e35afb607aed0f75385'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.588821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fe693c86d764e35afb607aed0f75385'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.589755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fe693c86d764e35afb607aed0f75385'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.610875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fe693c86d764e35afb607aed0f75385'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.632189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1040/2092] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:15 2022] DELETE /identity/v3/users/7fe693c86d764e35afb607aed0f75385 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:15.638907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f22fe-f49a-4e9a-9129-b239d47d8442 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.642243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f22fe-f49a-4e9a-9129-b239d47d8442 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.642243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f22fe-f49a-4e9a-9129-b239d47d8442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.676610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b81f22fe-f49a-4e9a-9129-b239d47d8442 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:15.678391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b81f22fe-f49a-4e9a-9129-b239d47d8442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:15.690468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:15.772409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1053/2093] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:15.786421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:15.788021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:15.825111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:15.846314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f22fe-f49a-4e9a-9129-b239d47d8442 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:15.847751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b81f22fe-f49a-4e9a-9129-b239d47d8442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:15.867911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f22fe-f49a-4e9a-9129-b239d47d8442 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.872824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:15.875455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.875455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.875455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] PATH_INFO: `/v3/projects/4fa5a4384ef740f4861c8ff5ee9f6456` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.876943 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa5a4384ef740f4861c8ff5ee9f6456)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.878903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.894644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9c98f-5ce9-4974-857a-943b16ba613a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.898374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec1256-789c-4728-a3f8-93b0d6b65729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:15.899179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec1256-789c-4728-a3f8-93b0d6b65729 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.899465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec1256-789c-4728-a3f8-93b0d6b65729 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.899743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec1256-789c-4728-a3f8-93b0d6b65729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.911269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec1256-789c-4728-a3f8-93b0d6b65729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.914281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec1256-789c-4728-a3f8-93b0d6b65729 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.926327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1041/2094] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:15.943343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa5a4384ef740f4861c8ff5ee9f6456', 'request_id': 'req-79c5955a-a0d7-4a89-849d-f0a7bd600c50'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.950696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa5a4384ef740f4861c8ff5ee9f6456', 'request_id': 'req-79c5955a-a0d7-4a89-849d-f0a7bd600c50'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.961656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa5a4384ef740f4861c8ff5ee9f6456', 'request_id': 'req-79c5955a-a0d7-4a89-849d-f0a7bd600c50'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.995497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] The token cache is being invalidate because project 4fa5a4384ef740f4861c8ff5ee9f6456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:15.995777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.996444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.998867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1054/2095] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:15 2022] DELETE /identity/v3/projects/4fa5a4384ef740f4861c8ff5ee9f6456 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:16.260763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec1256-789c-4728-a3f8-93b0d6b65729 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.262608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0fec1256-789c-4728-a3f8-93b0d6b65729 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:16.276689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec1256-789c-4728-a3f8-93b0d6b65729 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.337183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.337814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.338099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.338367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] PATH_INFO: `/v3/projects/5c5256840b8e44d08d798e51886828c6` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.339481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c5256840b8e44d08d798e51886828c6)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.341398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.390361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c5256840b8e44d08d798e51886828c6', 'request_id': 'req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.398188 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c5256840b8e44d08d798e51886828c6', 'request_id': 'req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.407537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c5256840b8e44d08d798e51886828c6', 'request_id': 'req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.418853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] The token cache is being invalidate because project 5c5256840b8e44d08d798e51886828c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:16.419255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.420285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.421926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1042/2096] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:16 2022] DELETE /identity/v3/projects/5c5256840b8e44d08d798e51886828c6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:16.426091 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.427605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:16.444315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5955a-a0d7-4a89-849d-f0a7bd600c50 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.503873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.504531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.505168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.506219 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] PATH_INFO: `/v3/users/5632ed8ec966440380472e659071eb18` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.507476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5632ed8ec966440380472e659071eb18)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.509431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.551910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:16.557184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5632ed8ec966440380472e659071eb18'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.565409 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5632ed8ec966440380472e659071eb18'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.573264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5632ed8ec966440380472e659071eb18'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.573965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5632ed8ec966440380472e659071eb18'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.586359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9a685049-6deb-4439-87f3-4822aa1c257d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5632ed8ec966440380472e659071eb18'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.593064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1055/2097] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:16 2022] DELETE /identity/v3/users/5632ed8ec966440380472e659071eb18 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:16.878565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.881291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:16.895073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6db79-32c3-42a7-a1a1-14ab3bf93287 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.948100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:16.972263 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.973446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.973882 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.974166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] PATH_INFO: `/v3/projects/ed3ff784822a4fd79348770f3d290e84` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.975360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed3ff784822a4fd79348770f3d290e84)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.977299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.031934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed3ff784822a4fd79348770f3d290e84', 'request_id': 'req-726c39bf-05d6-4f39-b464-82b54a21f8fb'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:17.039911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed3ff784822a4fd79348770f3d290e84', 'request_id': 'req-726c39bf-05d6-4f39-b464-82b54a21f8fb'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:17.050283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed3ff784822a4fd79348770f3d290e84', 'request_id': 'req-726c39bf-05d6-4f39-b464-82b54a21f8fb'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:17.062470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] The token cache is being invalidate because project ed3ff784822a4fd79348770f3d290e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:17.062896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:17.065112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-726c39bf-05d6-4f39-b464-82b54a21f8fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:17.066319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1043/2098] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:16 2022] DELETE /identity/v3/projects/ed3ff784822a4fd79348770f3d290e84 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:17.072506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aecc9a-9acd-4f5e-8041-5fe49724f3e7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.074143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aecc9a-9acd-4f5e-8041-5fe49724f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.074582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aecc9a-9acd-4f5e-8041-5fe49724f3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.099564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e0aecc9a-9acd-4f5e-8041-5fe49724f3e7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:17.099564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e0aecc9a-9acd-4f5e-8041-5fe49724f3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:17.162059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1056/2099] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:17.172391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158a703-21a1-44c4-af53-de2f0df026b8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.172667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158a703-21a1-44c4-af53-de2f0df026b8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.172936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158a703-21a1-44c4-af53-de2f0df026b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.200175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a158a703-21a1-44c4-af53-de2f0df026b8 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:17.201986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a158a703-21a1-44c4-af53-de2f0df026b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:17.211065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1044/2100] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:17.218980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aecc9a-9acd-4f5e-8041-5fe49724f3e7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.230836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aecc9a-9acd-4f5e-8041-5fe49724f3e7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.236695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05db17-f196-40d0-b309-e43b3db43f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.237783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05db17-f196-40d0-b309-e43b3db43f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.238732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05db17-f196-40d0-b309-e43b3db43f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.239321 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05db17-f196-40d0-b309-e43b3db43f33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.239797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05db17-f196-40d0-b309-e43b3db43f33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:17.240103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05db17-f196-40d0-b309-e43b3db43f33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.242061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05db17-f196-40d0-b309-e43b3db43f33 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.267071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1057/2101] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:17.271617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158a703-21a1-44c4-af53-de2f0df026b8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.282736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158a703-21a1-44c4-af53-de2f0df026b8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.287616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e72fdc-ac43-45b0-8ff4-0f3146aaf907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.288244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e72fdc-ac43-45b0-8ff4-0f3146aaf907 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.288521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e72fdc-ac43-45b0-8ff4-0f3146aaf907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.288789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e72fdc-ac43-45b0-8ff4-0f3146aaf907 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.289514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e72fdc-ac43-45b0-8ff4-0f3146aaf907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.291414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e72fdc-ac43-45b0-8ff4-0f3146aaf907 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.326248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1045/2102] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:17.327962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05db17-f196-40d0-b309-e43b3db43f33 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.339139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05db17-f196-40d0-b309-e43b3db43f33 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.418874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5d397-dd6f-4136-9349-c709c0d2287c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.419487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5d397-dd6f-4136-9349-c709c0d2287c admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.419772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5d397-dd6f-4136-9349-c709c0d2287c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.420906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5d397-dd6f-4136-9349-c709c0d2287c admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.421705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5d397-dd6f-4136-9349-c709c0d2287c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.423647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5d397-dd6f-4136-9349-c709c0d2287c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.446550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1058/2103] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:17.450302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e72fdc-ac43-45b0-8ff4-0f3146aaf907 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.464121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e72fdc-ac43-45b0-8ff4-0f3146aaf907 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.470865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc637ae2-282d-4778-a47f-75eb47d5379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.471494 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc637ae2-282d-4778-a47f-75eb47d5379d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.472792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc637ae2-282d-4778-a47f-75eb47d5379d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.473093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc637ae2-282d-4778-a47f-75eb47d5379d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.473766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc637ae2-282d-4778-a47f-75eb47d5379d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.475386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc637ae2-282d-4778-a47f-75eb47d5379d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.488807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1046/2104] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:17.492679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5d397-dd6f-4136-9349-c709c0d2287c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.506048 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5d397-dd6f-4136-9349-c709c0d2287c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.513667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70645f1c-6d0d-4918-855e-5c67ba4aeb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.515012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70645f1c-6d0d-4918-855e-5c67ba4aeb78 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.515527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70645f1c-6d0d-4918-855e-5c67ba4aeb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.515807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70645f1c-6d0d-4918-855e-5c67ba4aeb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.516463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70645f1c-6d0d-4918-855e-5c67ba4aeb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.518095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70645f1c-6d0d-4918-855e-5c67ba4aeb78 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.529524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1059/2105] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:17.533776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc637ae2-282d-4778-a47f-75eb47d5379d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.544065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc637ae2-282d-4778-a47f-75eb47d5379d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.549051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987558c-d22d-48df-933d-fe929b815ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.549732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987558c-d22d-48df-933d-fe929b815ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.550013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987558c-d22d-48df-933d-fe929b815ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.550313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987558c-d22d-48df-933d-fe929b815ba0 admin admin] PATH_INFO: `/v3/projects/d2ca9cf5f6244ad5bb17bcfd1a4c4573/users/7323455bb13e4cbb8c5b7c20975a3b59/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.567166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987558c-d22d-48df-933d-fe929b815ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ca9cf5f6244ad5bb17bcfd1a4c4573, user_id=7323455bb13e4cbb8c5b7c20975a3b59, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.569871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987558c-d22d-48df-933d-fe929b815ba0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.585130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1047/2106] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:17 2022] PUT /identity/v3/projects/d2ca9cf5f6244ad5bb17bcfd1a4c4573/users/7323455bb13e4cbb8c5b7c20975a3b59/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:17.590217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560172f9-6903-498e-af5e-c6c6684dcc02 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.590602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560172f9-6903-498e-af5e-c6c6684dcc02 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.590953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560172f9-6903-498e-af5e-c6c6684dcc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.612642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-560172f9-6903-498e-af5e-c6c6684dcc02 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:17.614006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-560172f9-6903-498e-af5e-c6c6684dcc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:17.633293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:17.678284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1060/2107] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:17.688601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987558c-d22d-48df-933d-fe929b815ba0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.690319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c987558c-d22d-48df-933d-fe929b815ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:17.752987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987558c-d22d-48df-933d-fe929b815ba0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.756674 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af4ef3-8d6e-4464-b3b3-1189767de6c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.757344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af4ef3-8d6e-4464-b3b3-1189767de6c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.757570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af4ef3-8d6e-4464-b3b3-1189767de6c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.757841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af4ef3-8d6e-4464-b3b3-1189767de6c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.767517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af4ef3-8d6e-4464-b3b3-1189767de6c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.768949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af4ef3-8d6e-4464-b3b3-1189767de6c6 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.779412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1048/2108] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:18.503746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e3842-4a33-4fd5-8d0d-c3a6a44db5f2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.504253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e3842-4a33-4fd5-8d0d-c3a6a44db5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.504349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e3842-4a33-4fd5-8d0d-c3a6a44db5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.544529 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7f0e3842-4a33-4fd5-8d0d-c3a6a44db5f2 None None] MFA Rules not processed for user `7323455bb13e4cbb8c5b7c20975a3b59`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:18.546558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7f0e3842-4a33-4fd5-8d0d-c3a6a44db5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:18.664736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1061/2109] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:18.669507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af4ef3-8d6e-4464-b3b3-1189767de6c6 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.696031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af4ef3-8d6e-4464-b3b3-1189767de6c6 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.705585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf29fe-e426-4644-9a5d-72a65fe7b2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.711235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf29fe-e426-4644-9a5d-72a65fe7b2cd admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.711783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf29fe-e426-4644-9a5d-72a65fe7b2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.712218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf29fe-e426-4644-9a5d-72a65fe7b2cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.713111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf29fe-e426-4644-9a5d-72a65fe7b2cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:18.716000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf29fe-e426-4644-9a5d-72a65fe7b2cd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:18.740217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:18.778631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1049/2110] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:18 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:18.783835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e3842-4a33-4fd5-8d0d-c3a6a44db5f2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.800214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e3842-4a33-4fd5-8d0d-c3a6a44db5f2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.922272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407a8d3-9943-4796-a475-22b538e67996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.923370 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407a8d3-9943-4796-a475-22b538e67996 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.923648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407a8d3-9943-4796-a475-22b538e67996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.923920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407a8d3-9943-4796-a475-22b538e67996 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.924585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407a8d3-9943-4796-a475-22b538e67996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:18.927432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407a8d3-9943-4796-a475-22b538e67996 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:18.950899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1062/2111] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:18 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:18.955732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf29fe-e426-4644-9a5d-72a65fe7b2cd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.966069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf29fe-e426-4644-9a5d-72a65fe7b2cd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.971791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34e19d-c23b-4882-9c70-b89005e5227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.972385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34e19d-c23b-4882-9c70-b89005e5227c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.972668 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34e19d-c23b-4882-9c70-b89005e5227c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.972943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34e19d-c23b-4882-9c70-b89005e5227c admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.973630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34e19d-c23b-4882-9c70-b89005e5227c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:18.975340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34e19d-c23b-4882-9c70-b89005e5227c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:18.986819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1050/2112] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:18.991133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407a8d3-9943-4796-a475-22b538e67996 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:19.001695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407a8d3-9943-4796-a475-22b538e67996 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:19.006278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e01fc-a5a2-4d86-9d62-5dd8e27137c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.006919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e01fc-a5a2-4d86-9d62-5dd8e27137c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.007195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e01fc-a5a2-4d86-9d62-5dd8e27137c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.007514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e01fc-a5a2-4d86-9d62-5dd8e27137c1 admin admin] PATH_INFO: `/v3/projects/90f92a8ecdcf4fa38d9108c6cc5d3c48/users/e92cc2e2d0b64a7e88524e667bb90778/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.023248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e01fc-a5a2-4d86-9d62-5dd8e27137c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f92a8ecdcf4fa38d9108c6cc5d3c48, user_id=e92cc2e2d0b64a7e88524e667bb90778, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.026526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e01fc-a5a2-4d86-9d62-5dd8e27137c1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.041289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1063/2113] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:18 2022] PUT /identity/v3/projects/90f92a8ecdcf4fa38d9108c6cc5d3c48/users/e92cc2e2d0b64a7e88524e667bb90778/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:19.045523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34e19d-c23b-4882-9c70-b89005e5227c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:19.056211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34e19d-c23b-4882-9c70-b89005e5227c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:19.123030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890d8d3-c209-4bd9-b1b5-c40e7eac9e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.124323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890d8d3-c209-4bd9-b1b5-c40e7eac9e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.124614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890d8d3-c209-4bd9-b1b5-c40e7eac9e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.124893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890d8d3-c209-4bd9-b1b5-c40e7eac9e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.125581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890d8d3-c209-4bd9-b1b5-c40e7eac9e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.127185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890d8d3-c209-4bd9-b1b5-c40e7eac9e29 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.143540 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1051/2114] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:19.147689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e01fc-a5a2-4d86-9d62-5dd8e27137c1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:19.160805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e01fc-a5a2-4d86-9d62-5dd8e27137c1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:19.165773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7426d-0ae4-4fd6-980d-41aba10402cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.166586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7426d-0ae4-4fd6-980d-41aba10402cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.166932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7426d-0ae4-4fd6-980d-41aba10402cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.167539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7426d-0ae4-4fd6-980d-41aba10402cf admin admin] PATH_INFO: `/v3/projects/90f92a8ecdcf4fa38d9108c6cc5d3c48/users/e92cc2e2d0b64a7e88524e667bb90778/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.173929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7426d-0ae4-4fd6-980d-41aba10402cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f92a8ecdcf4fa38d9108c6cc5d3c48, user_id=e92cc2e2d0b64a7e88524e667bb90778, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.175781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7426d-0ae4-4fd6-980d-41aba10402cf admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.187765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1064/2115] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:19 2022] PUT /identity/v3/projects/90f92a8ecdcf4fa38d9108c6cc5d3c48/users/e92cc2e2d0b64a7e88524e667bb90778/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:20.120386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d80012-fe5b-4b8c-bbfb-8c5a427fae36 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.120922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d80012-fe5b-4b8c-bbfb-8c5a427fae36 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.121033 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d80012-fe5b-4b8c-bbfb-8c5a427fae36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.151129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b6d80012-fe5b-4b8c-bbfb-8c5a427fae36 None None] MFA Rules not processed for user `e92cc2e2d0b64a7e88524e667bb90778`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:20.152465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b6d80012-fe5b-4b8c-bbfb-8c5a427fae36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:20.229233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1052/2116] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:20.246250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7426d-0ae4-4fd6-980d-41aba10402cf admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:20.248013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-34d7426d-0ae4-4fd6-980d-41aba10402cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:20.264085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:20.318903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7426d-0ae4-4fd6-980d-41aba10402cf admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:20.322415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0344a-7977-45b3-ae00-64e9ade109b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:20.323009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0344a-7977-45b3-ae00-64e9ade109b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.323281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0344a-7977-45b3-ae00-64e9ade109b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.323542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0344a-7977-45b3-ae00-64e9ade109b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.380654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0344a-7977-45b3-ae00-64e9ade109b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:20.382152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0344a-7977-45b3-ae00-64e9ade109b4 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:20.391143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1065/2117] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:21.761653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d80012-fe5b-4b8c-bbfb-8c5a427fae36 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.771826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d80012-fe5b-4b8c-bbfb-8c5a427fae36 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:21.775316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bf811-9eaf-41f5-ae24-945b3df67b2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:21.775901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bf811-9eaf-41f5-ae24-945b3df67b2a service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:21.776175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bf811-9eaf-41f5-ae24-945b3df67b2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:21.776441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bf811-9eaf-41f5-ae24-945b3df67b2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:21.786540 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bf811-9eaf-41f5-ae24-945b3df67b2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:21.788320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bf811-9eaf-41f5-ae24-945b3df67b2a service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:21.798040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1053/2118] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:23.652130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bf9a4-26c1-43be-9da1-89ed59257c4e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:23.652642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bf9a4-26c1-43be-9da1-89ed59257c4e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:23.652820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bf9a4-26c1-43be-9da1-89ed59257c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:23.676506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cc0bf9a4-26c1-43be-9da1-89ed59257c4e None None] MFA Rules not processed for user `fed877b0a569455597dfbb8b98ae0bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:23.677777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cc0bf9a4-26c1-43be-9da1-89ed59257c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:23.745671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1066/2119] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:28.371605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bf811-9eaf-41f5-ae24-945b3df67b2a service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:28.373730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1a0bf811-9eaf-41f5-ae24-945b3df67b2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:28.445200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:28.475681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bf811-9eaf-41f5-ae24-945b3df67b2a service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:28.481510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:28.484736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.485869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.486283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] PATH_INFO: `/v3/users/e9b5e1c1d85a4bd2b75d13bfaa7cc52d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.487729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b5e1c1d85a4bd2b75d13bfaa7cc52d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:28.490563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:28.566584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b5e1c1d85a4bd2b75d13bfaa7cc52d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:28.582688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b5e1c1d85a4bd2b75d13bfaa7cc52d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:28.583889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b5e1c1d85a4bd2b75d13bfaa7cc52d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:28.586158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b5e1c1d85a4bd2b75d13bfaa7cc52d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:28.620804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b5e1c1d85a4bd2b75d13bfaa7cc52d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:28.652069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1054/2120] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:28 2022] DELETE /identity/v3/users/e9b5e1c1d85a4bd2b75d13bfaa7cc52d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:28.656531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a878fbb-2ac5-4497-a461-98702cec8062 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.657017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a878fbb-2ac5-4497-a461-98702cec8062 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.657134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a878fbb-2ac5-4497-a461-98702cec8062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.687362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7a878fbb-2ac5-4497-a461-98702cec8062 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:28.689375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7a878fbb-2ac5-4497-a461-98702cec8062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:28.815909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1067/2121] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:28.853144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:28.854623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:28.933636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af974df-8ac5-4d86-8f3e-1b09b6824381 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:28.938305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212cc30-51ea-4ae5-9296-1def02dec927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:28.939019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212cc30-51ea-4ae5-9296-1def02dec927 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.939304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212cc30-51ea-4ae5-9296-1def02dec927 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.939577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212cc30-51ea-4ae5-9296-1def02dec927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.952541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212cc30-51ea-4ae5-9296-1def02dec927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:28.954276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212cc30-51ea-4ae5-9296-1def02dec927 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:28.985397 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1055/2122] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:29.607415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a878fbb-2ac5-4497-a461-98702cec8062 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.609029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7a878fbb-2ac5-4497-a461-98702cec8062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:29.627036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a878fbb-2ac5-4497-a461-98702cec8062 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.633710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.634864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.635637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.636456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] PATH_INFO: `/v3/projects/75874574a3f346b78a58218225f2e1b7` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.639536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] RBAC: Authorizing `identity:delete_project(project_id=75874574a3f346b78a58218225f2e1b7)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.642133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:29.643859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:29.693955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75874574a3f346b78a58218225f2e1b7', 'request_id': 'req-0df784c9-2ee5-4bd1-bda1-5400b086ecab'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:29.700171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75874574a3f346b78a58218225f2e1b7', 'request_id': 'req-0df784c9-2ee5-4bd1-bda1-5400b086ecab'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:29.709838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75874574a3f346b78a58218225f2e1b7', 'request_id': 'req-0df784c9-2ee5-4bd1-bda1-5400b086ecab'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:29.721998 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] The token cache is being invalidate because project 75874574a3f346b78a58218225f2e1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:29.722294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:29.722970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0df784c9-2ee5-4bd1-bda1-5400b086ecab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:29.728122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1068/2123] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:29 2022] DELETE /identity/v3/projects/75874574a3f346b78a58218225f2e1b7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:29.729877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802f447-08e3-48ce-bf49-cdf79cef6b0e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.730334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802f447-08e3-48ce-bf49-cdf79cef6b0e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.730424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802f447-08e3-48ce-bf49-cdf79cef6b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.751497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1802f447-08e3-48ce-bf49-cdf79cef6b0e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:29.752949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1802f447-08e3-48ce-bf49-cdf79cef6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:29.823598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1056/2124] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:29.831872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce0c37-03de-4d87-bd1c-9a2a80b061b6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.832220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce0c37-03de-4d87-bd1c-9a2a80b061b6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.832500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce0c37-03de-4d87-bd1c-9a2a80b061b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.883346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-13ce0c37-03de-4d87-bd1c-9a2a80b061b6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:29.884811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-13ce0c37-03de-4d87-bd1c-9a2a80b061b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:29.900002 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1069/2125] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:29.901246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802f447-08e3-48ce-bf49-cdf79cef6b0e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.914380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802f447-08e3-48ce-bf49-cdf79cef6b0e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.924995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eb35e-f589-4149-b228-5d5f737bd896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.925825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eb35e-f589-4149-b228-5d5f737bd896 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.926177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eb35e-f589-4149-b228-5d5f737bd896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.926520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eb35e-f589-4149-b228-5d5f737bd896 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.927271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4eb35e-f589-4149-b228-5d5f737bd896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:29.928028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4eb35e-f589-4149-b228-5d5f737bd896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.929774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4eb35e-f589-4149-b228-5d5f737bd896 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:29.937735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:29.949044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1057/2126] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:29.953483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce0c37-03de-4d87-bd1c-9a2a80b061b6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.963887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce0c37-03de-4d87-bd1c-9a2a80b061b6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.974624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c7a9-ad41-4f88-953a-9ad835e606a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.975536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c7a9-ad41-4f88-953a-9ad835e606a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.975962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c7a9-ad41-4f88-953a-9ad835e606a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.976379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c7a9-ad41-4f88-953a-9ad835e606a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.977520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c7a9-ad41-4f88-953a-9ad835e606a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.984464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c7a9-ad41-4f88-953a-9ad835e606a2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:30.061842 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1070/2127] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:29 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:30.065984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eb35e-f589-4149-b228-5d5f737bd896 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:30.083100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eb35e-f589-4149-b228-5d5f737bd896 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:30.184209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30747b-bf86-45d2-b8eb-7322160e7a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:30.184916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30747b-bf86-45d2-b8eb-7322160e7a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.186026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30747b-bf86-45d2-b8eb-7322160e7a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.186157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30747b-bf86-45d2-b8eb-7322160e7a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.186865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30747b-bf86-45d2-b8eb-7322160e7a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:30.188905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30747b-bf86-45d2-b8eb-7322160e7a0e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:30.235561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1058/2128] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:30 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:30.241140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c7a9-ad41-4f88-953a-9ad835e606a2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:30.268295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c7a9-ad41-4f88-953a-9ad835e606a2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:30.277786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5b46-57c0-49c4-bdab-23bc1c601bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:30.278738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5b46-57c0-49c4-bdab-23bc1c601bae admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.279524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5b46-57c0-49c4-bdab-23bc1c601bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.281298 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a5b46-57c0-49c4-bdab-23bc1c601bae admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.282094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a5b46-57c0-49c4-bdab-23bc1c601bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:30.283963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a5b46-57c0-49c4-bdab-23bc1c601bae admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:30.298565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1071/2129] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:30.300072 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30747b-bf86-45d2-b8eb-7322160e7a0e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:30.313029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30747b-bf86-45d2-b8eb-7322160e7a0e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:30.318885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63febb6-cb51-4c68-b809-c599cf50f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:30.319726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63febb6-cb51-4c68-b809-c599cf50f2bf admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.320010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63febb6-cb51-4c68-b809-c599cf50f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.320279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63febb6-cb51-4c68-b809-c599cf50f2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.321215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63febb6-cb51-4c68-b809-c599cf50f2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:30.322978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63febb6-cb51-4c68-b809-c599cf50f2bf admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:30.344853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1059/2130] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:30.346215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5b46-57c0-49c4-bdab-23bc1c601bae admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:30.360096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a5b46-57c0-49c4-bdab-23bc1c601bae admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:30.365622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be4a43-a0ec-4dd2-a585-f094b75fa54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:30.366478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be4a43-a0ec-4dd2-a585-f094b75fa54e admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.366755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be4a43-a0ec-4dd2-a585-f094b75fa54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.367050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be4a43-a0ec-4dd2-a585-f094b75fa54e admin admin] PATH_INFO: `/v3/projects/630d4584c0ec4311b247a57ad19a5c75/users/b35bab67a0cd45439f84bf321e6a9662/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.384017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be4a43-a0ec-4dd2-a585-f094b75fa54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=630d4584c0ec4311b247a57ad19a5c75, user_id=b35bab67a0cd45439f84bf321e6a9662, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:30.386846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be4a43-a0ec-4dd2-a585-f094b75fa54e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:30.414120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1072/2131] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:30 2022] PUT /identity/v3/projects/630d4584c0ec4311b247a57ad19a5c75/users/b35bab67a0cd45439f84bf321e6a9662/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:30.425023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630208c0-e961-4e5d-af9f-ece53b0b2b74 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.425357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630208c0-e961-4e5d-af9f-ece53b0b2b74 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.425658 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630208c0-e961-4e5d-af9f-ece53b0b2b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.473609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-630208c0-e961-4e5d-af9f-ece53b0b2b74 None None] MFA Rules not processed for user `b35bab67a0cd45439f84bf321e6a9662`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:30.475315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-630208c0-e961-4e5d-af9f-ece53b0b2b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:30.559963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be4a43-a0ec-4dd2-a585-f094b75fa54e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:30.562309 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-00be4a43-a0ec-4dd2-a585-f094b75fa54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:30.602526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1060/2132] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:30.616284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630208c0-e961-4e5d-af9f-ece53b0b2b74 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:30.617831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-630208c0-e961-4e5d-af9f-ece53b0b2b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:30.661835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be4a43-a0ec-4dd2-a585-f094b75fa54e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:30.666806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:30.667637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.668127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.668634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] PATH_INFO: `/v3/users/e4cc89946c2e45a0a75eef16cb042758` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.670838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4cc89946c2e45a0a75eef16cb042758)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:30.672971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:30.696405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630208c0-e961-4e5d-af9f-ece53b0b2b74 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:30.700571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838014f-4d97-418a-93e4-70d3710b2dab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:30.701334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838014f-4d97-418a-93e4-70d3710b2dab service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.701653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838014f-4d97-418a-93e4-70d3710b2dab service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.701939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838014f-4d97-418a-93e4-70d3710b2dab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.710169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:30.718779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838014f-4d97-418a-93e4-70d3710b2dab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:30.720413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838014f-4d97-418a-93e4-70d3710b2dab service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:30.730811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1061/2133] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:30.736795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4cc89946c2e45a0a75eef16cb042758'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:30.748854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4cc89946c2e45a0a75eef16cb042758'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:30.759594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4cc89946c2e45a0a75eef16cb042758'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:30.760441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4cc89946c2e45a0a75eef16cb042758'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:30.780371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-34856050-e504-40eb-a3eb-67f1946c86fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4cc89946c2e45a0a75eef16cb042758'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:30.793333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1073/2134] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:30 2022] DELETE /identity/v3/users/e4cc89946c2e45a0a75eef16cb042758 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:31.003209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838014f-4d97-418a-93e4-70d3710b2dab service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.038809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838014f-4d97-418a-93e4-70d3710b2dab service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.046391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.132373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.133035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.133343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.133739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] PATH_INFO: `/v3/projects/0c0f91770bb14bb6920fc277936c189e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.135227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0f91770bb14bb6920fc277936c189e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.138943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.192789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0f91770bb14bb6920fc277936c189e', 'request_id': 'req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:31.199052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0f91770bb14bb6920fc277936c189e', 'request_id': 'req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:31.207656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0f91770bb14bb6920fc277936c189e', 'request_id': 'req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:31.220240 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] The token cache is being invalidate because project 0c0f91770bb14bb6920fc277936c189e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:31.220551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:31.221260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fa3c6887-e15e-4bb2-abcd-e5e708e11d08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:31.222785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1062/2135] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:31 2022] DELETE /identity/v3/projects/0c0f91770bb14bb6920fc277936c189e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:31.227695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db4c00-1ee0-4253-950d-7f42dee3bd99 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.227986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db4c00-1ee0-4253-950d-7f42dee3bd99 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.228367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db4c00-1ee0-4253-950d-7f42dee3bd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.252440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c6db4c00-1ee0-4253-950d-7f42dee3bd99 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:31.253699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c6db4c00-1ee0-4253-950d-7f42dee3bd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:31.324061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1074/2136] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:31.332511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07b7d1-ed56-435d-b5e2-1e70a3f08525 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.332800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07b7d1-ed56-435d-b5e2-1e70a3f08525 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.333105 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07b7d1-ed56-435d-b5e2-1e70a3f08525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.353134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8f07b7d1-ed56-435d-b5e2-1e70a3f08525 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:31.354351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8f07b7d1-ed56-435d-b5e2-1e70a3f08525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:31.364414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1063/2137] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:31.368729 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db4c00-1ee0-4253-950d-7f42dee3bd99 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.379902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db4c00-1ee0-4253-950d-7f42dee3bd99 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.384537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abd4ba-b10b-41f7-b887-e2c6606bfa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.385165 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abd4ba-b10b-41f7-b887-e2c6606bfa57 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.385465 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abd4ba-b10b-41f7-b887-e2c6606bfa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.385738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abd4ba-b10b-41f7-b887-e2c6606bfa57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.386381 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abd4ba-b10b-41f7-b887-e2c6606bfa57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:31.386680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abd4ba-b10b-41f7-b887-e2c6606bfa57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.388459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abd4ba-b10b-41f7-b887-e2c6606bfa57 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.400222 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1075/2138] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:31.404285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07b7d1-ed56-435d-b5e2-1e70a3f08525 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.414320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07b7d1-ed56-435d-b5e2-1e70a3f08525 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.419211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659513a-1525-456c-a2e2-f97b40f8aff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.420421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659513a-1525-456c-a2e2-f97b40f8aff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.420800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659513a-1525-456c-a2e2-f97b40f8aff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.421089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659513a-1525-456c-a2e2-f97b40f8aff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.421737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659513a-1525-456c-a2e2-f97b40f8aff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.423756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659513a-1525-456c-a2e2-f97b40f8aff8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.453627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1064/2139] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:31 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:31.458430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abd4ba-b10b-41f7-b887-e2c6606bfa57 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.470228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abd4ba-b10b-41f7-b887-e2c6606bfa57 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.541993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517960d-78de-4b69-80f4-048cd9c62d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.542820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517960d-78de-4b69-80f4-048cd9c62d26 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.543268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517960d-78de-4b69-80f4-048cd9c62d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.543684 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517960d-78de-4b69-80f4-048cd9c62d26 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.546010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517960d-78de-4b69-80f4-048cd9c62d26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.548182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517960d-78de-4b69-80f4-048cd9c62d26 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.574594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1076/2140] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:31 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:31.579028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659513a-1525-456c-a2e2-f97b40f8aff8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.591163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659513a-1525-456c-a2e2-f97b40f8aff8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.596200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c49285-0a81-4db3-a6c0-0b4b0dfeeea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.596796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c49285-0a81-4db3-a6c0-0b4b0dfeeea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.597165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c49285-0a81-4db3-a6c0-0b4b0dfeeea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.597361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c49285-0a81-4db3-a6c0-0b4b0dfeeea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.597998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c49285-0a81-4db3-a6c0-0b4b0dfeeea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.599562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c49285-0a81-4db3-a6c0-0b4b0dfeeea2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.610333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1065/2141] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:31.614299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517960d-78de-4b69-80f4-048cd9c62d26 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.625407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517960d-78de-4b69-80f4-048cd9c62d26 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.630012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d703a0-c67c-4fd1-aa89-f7402f21109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.630600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d703a0-c67c-4fd1-aa89-f7402f21109a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.630878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d703a0-c67c-4fd1-aa89-f7402f21109a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.631145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d703a0-c67c-4fd1-aa89-f7402f21109a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.631778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d703a0-c67c-4fd1-aa89-f7402f21109a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.633403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d703a0-c67c-4fd1-aa89-f7402f21109a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.643576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1077/2142] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:31.647874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c49285-0a81-4db3-a6c0-0b4b0dfeeea2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.658414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c49285-0a81-4db3-a6c0-0b4b0dfeeea2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.663095 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95735b31-ee2e-40c4-a8b8-9e3b00fb704e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.663753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95735b31-ee2e-40c4-a8b8-9e3b00fb704e admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.664030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95735b31-ee2e-40c4-a8b8-9e3b00fb704e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.664299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95735b31-ee2e-40c4-a8b8-9e3b00fb704e admin admin] PATH_INFO: `/v3/projects/92ca392a802e4d1eb0f3078a9c17b78d/users/f7ce3a318fcc408eaaab38f28e9751b8/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.680272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95735b31-ee2e-40c4-a8b8-9e3b00fb704e admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ca392a802e4d1eb0f3078a9c17b78d, user_id=f7ce3a318fcc408eaaab38f28e9751b8, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.682921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95735b31-ee2e-40c4-a8b8-9e3b00fb704e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.695954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1066/2143] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:31 2022] PUT /identity/v3/projects/92ca392a802e4d1eb0f3078a9c17b78d/users/f7ce3a318fcc408eaaab38f28e9751b8/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:31.706166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34c07d-04d8-46c2-bc98-f148d2371659 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.706451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34c07d-04d8-46c2-bc98-f148d2371659 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.706726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34c07d-04d8-46c2-bc98-f148d2371659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.727452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.736130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8a34c07d-04d8-46c2-bc98-f148d2371659 None None] MFA Rules not processed for user `f7ce3a318fcc408eaaab38f28e9751b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:31.737991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8a34c07d-04d8-46c2-bc98-f148d2371659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:31.802947 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1078/2144] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:31.814065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95735b31-ee2e-40c4-a8b8-9e3b00fb704e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.815736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-95735b31-ee2e-40c4-a8b8-9e3b00fb704e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:31.851921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.894669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95735b31-ee2e-40c4-a8b8-9e3b00fb704e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.900278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddb033-b391-48ae-a31b-9d9b5639631e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.900907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb033-b391-48ae-a31b-9d9b5639631e service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.901213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb033-b391-48ae-a31b-9d9b5639631e service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.901479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb033-b391-48ae-a31b-9d9b5639631e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.910289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddb033-b391-48ae-a31b-9d9b5639631e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.911854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddb033-b391-48ae-a31b-9d9b5639631e service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.922649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1067/2145] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:32.269420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34c07d-04d8-46c2-bc98-f148d2371659 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:32.340684 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34c07d-04d8-46c2-bc98-f148d2371659 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:32.346101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:32.346737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.347021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.347293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] PATH_INFO: `/v3/users/f7ce3a318fcc408eaaab38f28e9751b8` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.348783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ce3a318fcc408eaaab38f28e9751b8)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:32.350896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:32.402716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ce3a318fcc408eaaab38f28e9751b8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:32.411303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ce3a318fcc408eaaab38f28e9751b8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:32.421094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ce3a318fcc408eaaab38f28e9751b8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:32.422080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ce3a318fcc408eaaab38f28e9751b8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:32.439476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ce3a318fcc408eaaab38f28e9751b8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:32.457137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1079/2146] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:32 2022] DELETE /identity/v3/users/f7ce3a318fcc408eaaab38f28e9751b8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:32.457559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc083626-5519-44a4-8b94-85ed0fdf4305 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.457850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc083626-5519-44a4-8b94-85ed0fdf4305 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.458129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc083626-5519-44a4-8b94-85ed0fdf4305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.479198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cc083626-5519-44a4-8b94-85ed0fdf4305 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:32.480328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cc083626-5519-44a4-8b94-85ed0fdf4305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:32.551564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1068/2147] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:32.563594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:32.566011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:32.602474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:32.664771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b07b5-2397-43d4-9b0f-40de49fca4c7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:32.668388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032df4a-8486-4ba7-a629-cbceb4a273fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:32.668999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032df4a-8486-4ba7-a629-cbceb4a273fb service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.669306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032df4a-8486-4ba7-a629-cbceb4a273fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.670554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032df4a-8486-4ba7-a629-cbceb4a273fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.680174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032df4a-8486-4ba7-a629-cbceb4a273fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:32.681727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032df4a-8486-4ba7-a629-cbceb4a273fb service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:32.693212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1080/2148] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:33.040627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc083626-5519-44a4-8b94-85ed0fdf4305 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.042174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cc083626-5519-44a4-8b94-85ed0fdf4305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.060544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc083626-5519-44a4-8b94-85ed0fdf4305 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.066792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.067491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.067946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.068245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] PATH_INFO: `/v3/projects/92ca392a802e4d1eb0f3078a9c17b78d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.069445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ca392a802e4d1eb0f3078a9c17b78d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.071365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.134616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ca392a802e4d1eb0f3078a9c17b78d', 'request_id': 'req-5a7de488-5d53-4689-93fe-39989570e927'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.143796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ca392a802e4d1eb0f3078a9c17b78d', 'request_id': 'req-5a7de488-5d53-4689-93fe-39989570e927'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.162824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ca392a802e4d1eb0f3078a9c17b78d', 'request_id': 'req-5a7de488-5d53-4689-93fe-39989570e927'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.179577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] The token cache is being invalidate because project 92ca392a802e4d1eb0f3078a9c17b78d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:33.179686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.180502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5a7de488-5d53-4689-93fe-39989570e927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.183166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1069/2149] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:33 2022] DELETE /identity/v3/projects/92ca392a802e4d1eb0f3078a9c17b78d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:33.190745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b610bf-093d-4098-a8c5-df458dd67e69 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.191334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b610bf-093d-4098-a8c5-df458dd67e69 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.191334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b610bf-093d-4098-a8c5-df458dd67e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.213016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-43b610bf-093d-4098-a8c5-df458dd67e69 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:33.214277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-43b610bf-093d-4098-a8c5-df458dd67e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.309146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1081/2150] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:33.321913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6efc45-7962-4113-8a6a-6047e151c31a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.322409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6efc45-7962-4113-8a6a-6047e151c31a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.322782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6efc45-7962-4113-8a6a-6047e151c31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.356419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cb6efc45-7962-4113-8a6a-6047e151c31a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:33.358011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cb6efc45-7962-4113-8a6a-6047e151c31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.366604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1070/2151] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:33.370882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b610bf-093d-4098-a8c5-df458dd67e69 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.382109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b610bf-093d-4098-a8c5-df458dd67e69 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.390603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d80d7a-e56b-4774-b4fe-a982d82e90b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.391219 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d80d7a-e56b-4774-b4fe-a982d82e90b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.391495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d80d7a-e56b-4774-b4fe-a982d82e90b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.391827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d80d7a-e56b-4774-b4fe-a982d82e90b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.392485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d80d7a-e56b-4774-b4fe-a982d82e90b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:33.392783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d80d7a-e56b-4774-b4fe-a982d82e90b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.394478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d80d7a-e56b-4774-b4fe-a982d82e90b3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.406629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1082/2152] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:33.411181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6efc45-7962-4113-8a6a-6047e151c31a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.420828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6efc45-7962-4113-8a6a-6047e151c31a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.425766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c1c38-f2b1-45cc-a8c5-f4ef9d53fa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.426352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c1c38-f2b1-45cc-a8c5-f4ef9d53fa0a admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.426632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c1c38-f2b1-45cc-a8c5-f4ef9d53fa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.426955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c1c38-f2b1-45cc-a8c5-f4ef9d53fa0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.427661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c1c38-f2b1-45cc-a8c5-f4ef9d53fa0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.429579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c1c38-f2b1-45cc-a8c5-f4ef9d53fa0a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.457883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1071/2153] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:33.461919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d80d7a-e56b-4774-b4fe-a982d82e90b3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.472181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d80d7a-e56b-4774-b4fe-a982d82e90b3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.531075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ae7e7-0e48-48ca-a055-7ae6337c6cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.531588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ae7e7-0e48-48ca-a055-7ae6337c6cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.531903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ae7e7-0e48-48ca-a055-7ae6337c6cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.532144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ae7e7-0e48-48ca-a055-7ae6337c6cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.533007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ae7e7-0e48-48ca-a055-7ae6337c6cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.534974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ae7e7-0e48-48ca-a055-7ae6337c6cd3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.557454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1083/2154] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:33.562217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c1c38-f2b1-45cc-a8c5-f4ef9d53fa0a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.573135 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c1c38-f2b1-45cc-a8c5-f4ef9d53fa0a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.578551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b788f4-4146-4f18-a721-470aa097572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.579220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b788f4-4146-4f18-a721-470aa097572a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.579568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b788f4-4146-4f18-a721-470aa097572a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.579846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b788f4-4146-4f18-a721-470aa097572a admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.580497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b788f4-4146-4f18-a721-470aa097572a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.582203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b788f4-4146-4f18-a721-470aa097572a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.592318 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1072/2155] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:33.596316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ae7e7-0e48-48ca-a055-7ae6337c6cd3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.606415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ae7e7-0e48-48ca-a055-7ae6337c6cd3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.611757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d3600-7d47-4a8b-bccb-6226cf3ba90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.612359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d3600-7d47-4a8b-bccb-6226cf3ba90e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.612624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d3600-7d47-4a8b-bccb-6226cf3ba90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.612895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d3600-7d47-4a8b-bccb-6226cf3ba90e admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.613564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d3600-7d47-4a8b-bccb-6226cf3ba90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.615145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d3600-7d47-4a8b-bccb-6226cf3ba90e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.626891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1084/2156] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:33.630928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b788f4-4146-4f18-a721-470aa097572a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.636004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.642815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b788f4-4146-4f18-a721-470aa097572a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.648572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03cab7-dc1f-4004-aa7b-0e5479a66d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.649258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03cab7-dc1f-4004-aa7b-0e5479a66d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.649537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03cab7-dc1f-4004-aa7b-0e5479a66d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.649811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03cab7-dc1f-4004-aa7b-0e5479a66d39 admin admin] PATH_INFO: `/v3/projects/156335eaf6384d568d74957a1b4e4f4b/users/a8b7acb2dcc6496ca322c00793443d0f/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.664790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03cab7-dc1f-4004-aa7b-0e5479a66d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156335eaf6384d568d74957a1b4e4f4b, user_id=a8b7acb2dcc6496ca322c00793443d0f, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.667482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03cab7-dc1f-4004-aa7b-0e5479a66d39 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.680735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1073/2157] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:33 2022] PUT /identity/v3/projects/156335eaf6384d568d74957a1b4e4f4b/users/a8b7acb2dcc6496ca322c00793443d0f/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:33.691511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2c404-d0ed-4fbd-b8f5-0b4451c99f2a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.691795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2c404-d0ed-4fbd-b8f5-0b4451c99f2a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.692067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2c404-d0ed-4fbd-b8f5-0b4451c99f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.720455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-fcf2c404-d0ed-4fbd-b8f5-0b4451c99f2a None None] MFA Rules not processed for user `a8b7acb2dcc6496ca322c00793443d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:33.721688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fcf2c404-d0ed-4fbd-b8f5-0b4451c99f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.738835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.788758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1085/2158] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:33.793313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03cab7-dc1f-4004-aa7b-0e5479a66d39 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.854784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03cab7-dc1f-4004-aa7b-0e5479a66d39 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.859970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bfcb8-fc0d-4a59-bca3-1937bcde1049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.860568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bfcb8-fc0d-4a59-bca3-1937bcde1049 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.860848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bfcb8-fc0d-4a59-bca3-1937bcde1049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.861148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bfcb8-fc0d-4a59-bca3-1937bcde1049 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.861802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bfcb8-fc0d-4a59-bca3-1937bcde1049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.863687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bfcb8-fc0d-4a59-bca3-1937bcde1049 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.889802 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1074/2159] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:33.893956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2c404-d0ed-4fbd-b8f5-0b4451c99f2a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.904634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2c404-d0ed-4fbd-b8f5-0b4451c99f2a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.961982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14123ace-8111-4d19-b0c7-6a70cba85727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.962621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14123ace-8111-4d19-b0c7-6a70cba85727 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.963037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14123ace-8111-4d19-b0c7-6a70cba85727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.966079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14123ace-8111-4d19-b0c7-6a70cba85727 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.966079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14123ace-8111-4d19-b0c7-6a70cba85727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.966387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14123ace-8111-4d19-b0c7-6a70cba85727 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.968375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bfcb8-fc0d-4a59-bca3-1937bcde1049 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.970092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b00bfcb8-fc0d-4a59-bca3-1937bcde1049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.984297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bfcb8-fc0d-4a59-bca3-1937bcde1049 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.989099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.989742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.990025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.990297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] PATH_INFO: `/v3/users/b35bab67a0cd45439f84bf321e6a9662` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.993717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1086/2160] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:33.993821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35bab67a0cd45439f84bf321e6a9662)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.993821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.994671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14123ace-8111-4d19-b0c7-6a70cba85727 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.006687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14123ace-8111-4d19-b0c7-6a70cba85727 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.013047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89127d3f-bacf-44bf-b09a-54875fca54c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.013738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127d3f-bacf-44bf-b09a-54875fca54c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.014020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127d3f-bacf-44bf-b09a-54875fca54c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.014555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127d3f-bacf-44bf-b09a-54875fca54c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.015130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89127d3f-bacf-44bf-b09a-54875fca54c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.016718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89127d3f-bacf-44bf-b09a-54875fca54c4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.028260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1087/2161] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:34.032817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89127d3f-bacf-44bf-b09a-54875fca54c4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.042446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89127d3f-bacf-44bf-b09a-54875fca54c4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.044576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35bab67a0cd45439f84bf321e6a9662'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.048557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5d242-d662-4839-aadb-1c4898b63735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.049171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5d242-d662-4839-aadb-1c4898b63735 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.049446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5d242-d662-4839-aadb-1c4898b63735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.049712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5d242-d662-4839-aadb-1c4898b63735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.050375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5d242-d662-4839-aadb-1c4898b63735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.051954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5d242-d662-4839-aadb-1c4898b63735 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.054749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35bab67a0cd45439f84bf321e6a9662'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.055398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35bab67a0cd45439f84bf321e6a9662'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.056206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35bab67a0cd45439f84bf321e6a9662'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.064868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1088/2162] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:34.069015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5d242-d662-4839-aadb-1c4898b63735 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.070918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-47d5d242-d662-4839-aadb-1c4898b63735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.071154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7a94862c-492a-43eb-a7eb-1cdd05aa28b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35bab67a0cd45439f84bf321e6a9662'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.085828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5d242-d662-4839-aadb-1c4898b63735 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.090459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503269e-4eed-4c47-ae09-e6ec6727dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.091119 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503269e-4eed-4c47-ae09-e6ec6727dcf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.091399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503269e-4eed-4c47-ae09-e6ec6727dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.091672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503269e-4eed-4c47-ae09-e6ec6727dcf9 admin admin] PATH_INFO: `/v3/projects/9fc0f43ed2e54de0a0cd62394b598409/users/9ee2031646304d1293cbb93b25dc4161/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.095391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1075/2163] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:33 2022] DELETE /identity/v3/users/b35bab67a0cd45439f84bf321e6a9662 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:34.097013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa43809-7643-4c9c-afe3-ae50dd9d10c6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.097013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa43809-7643-4c9c-afe3-ae50dd9d10c6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.097246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa43809-7643-4c9c-afe3-ae50dd9d10c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.108350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503269e-4eed-4c47-ae09-e6ec6727dcf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc0f43ed2e54de0a0cd62394b598409, user_id=9ee2031646304d1293cbb93b25dc4161, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.110997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503269e-4eed-4c47-ae09-e6ec6727dcf9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.117505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-eaa43809-7643-4c9c-afe3-ae50dd9d10c6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:34.118722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-eaa43809-7643-4c9c-afe3-ae50dd9d10c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.127734 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1089/2164] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:34 2022] PUT /identity/v3/projects/9fc0f43ed2e54de0a0cd62394b598409/users/9ee2031646304d1293cbb93b25dc4161/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:34.138487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07425499-4a0e-4d24-a8e3-1407a9026937 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.139363 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07425499-4a0e-4d24-a8e3-1407a9026937 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.139617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07425499-4a0e-4d24-a8e3-1407a9026937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.159161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.171191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-07425499-4a0e-4d24-a8e3-1407a9026937 None None] MFA Rules not processed for user `9ee2031646304d1293cbb93b25dc4161`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:34.172177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-07425499-4a0e-4d24-a8e3-1407a9026937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.201625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1076/2165] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:34.211066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa43809-7643-4c9c-afe3-ae50dd9d10c6 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.212408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-eaa43809-7643-4c9c-afe3-ae50dd9d10c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.225517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.250111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1090/2166] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:34.254476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07425499-4a0e-4d24-a8e3-1407a9026937 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.270638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07425499-4a0e-4d24-a8e3-1407a9026937 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.275349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a0957-0af1-4531-b983-354764de5bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.275973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a0957-0af1-4531-b983-354764de5bba admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.276249 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a0957-0af1-4531-b983-354764de5bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.276517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a0957-0af1-4531-b983-354764de5bba admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.276661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa43809-7643-4c9c-afe3-ae50dd9d10c6 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.277188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a0957-0af1-4531-b983-354764de5bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.279086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a0957-0af1-4531-b983-354764de5bba admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.280943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cb8bb-f6c6-45e6-891b-1da3328f277c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.281590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cb8bb-f6c6-45e6-891b-1da3328f277c service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.281870 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cb8bb-f6c6-45e6-891b-1da3328f277c service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.282141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cb8bb-f6c6-45e6-891b-1da3328f277c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.291720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cb8bb-f6c6-45e6-891b-1da3328f277c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.293205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cb8bb-f6c6-45e6-891b-1da3328f277c service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.304459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1077/2167] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:34.310181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1091/2168] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:34.314346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cb8bb-f6c6-45e6-891b-1da3328f277c service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.324689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cb8bb-f6c6-45e6-891b-1da3328f277c service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.408907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80162d-3d0d-4f93-9534-950687767a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.410073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80162d-3d0d-4f93-9534-950687767a20 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.410509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80162d-3d0d-4f93-9534-950687767a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.410786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80162d-3d0d-4f93-9534-950687767a20 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.412348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80162d-3d0d-4f93-9534-950687767a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.414283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80162d-3d0d-4f93-9534-950687767a20 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.472801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1078/2169] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:34.474247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a0957-0af1-4531-b983-354764de5bba admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.490954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a0957-0af1-4531-b983-354764de5bba admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.499339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc4dfd-02ba-4286-b2da-321337687b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.500065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc4dfd-02ba-4286-b2da-321337687b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.500391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc4dfd-02ba-4286-b2da-321337687b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.500606 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc4dfd-02ba-4286-b2da-321337687b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.501462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc4dfd-02ba-4286-b2da-321337687b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.503068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc4dfd-02ba-4286-b2da-321337687b57 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.521546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1092/2170] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:34.522772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80162d-3d0d-4f93-9534-950687767a20 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.535920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80162d-3d0d-4f93-9534-950687767a20 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.540425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8f3d0-5371-4518-bb4f-6fa84c0085dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.541112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f3d0-5371-4518-bb4f-6fa84c0085dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.548330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f3d0-5371-4518-bb4f-6fa84c0085dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.549201 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f3d0-5371-4518-bb4f-6fa84c0085dc admin admin] PATH_INFO: `/v3/projects/3cf1c667947e45cdb95ccad299a6ecc3/users/67e9a0cec46f44a9b7cc709ef47fd664/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.574909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8f3d0-5371-4518-bb4f-6fa84c0085dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf1c667947e45cdb95ccad299a6ecc3, user_id=67e9a0cec46f44a9b7cc709ef47fd664, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.577869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8f3d0-5371-4518-bb4f-6fa84c0085dc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.599669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1079/2171] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:34 2022] PUT /identity/v3/projects/3cf1c667947e45cdb95ccad299a6ecc3/users/67e9a0cec46f44a9b7cc709ef47fd664/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:34.603520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc4dfd-02ba-4286-b2da-321337687b57 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.615640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc4dfd-02ba-4286-b2da-321337687b57 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.707847 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873ca8d-5994-4db8-b4f7-f3ce17e6c89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.708769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873ca8d-5994-4db8-b4f7-f3ce17e6c89e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.709041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873ca8d-5994-4db8-b4f7-f3ce17e6c89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.709838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873ca8d-5994-4db8-b4f7-f3ce17e6c89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.710046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873ca8d-5994-4db8-b4f7-f3ce17e6c89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.712153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873ca8d-5994-4db8-b4f7-f3ce17e6c89e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.725128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1093/2172] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:34.729366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8f3d0-5371-4518-bb4f-6fa84c0085dc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.741389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8f3d0-5371-4518-bb4f-6fa84c0085dc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.746856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2dbdd-425d-49f0-b04f-b78c23df29e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.749002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2dbdd-425d-49f0-b04f-b78c23df29e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.751647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2dbdd-425d-49f0-b04f-b78c23df29e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.752287 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2dbdd-425d-49f0-b04f-b78c23df29e9 admin admin] PATH_INFO: `/v3/projects/3cf1c667947e45cdb95ccad299a6ecc3/users/67e9a0cec46f44a9b7cc709ef47fd664/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.760475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2dbdd-425d-49f0-b04f-b78c23df29e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf1c667947e45cdb95ccad299a6ecc3, user_id=67e9a0cec46f44a9b7cc709ef47fd664, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.765061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2dbdd-425d-49f0-b04f-b78c23df29e9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.777172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1080/2173] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:34 2022] PUT /identity/v3/projects/3cf1c667947e45cdb95ccad299a6ecc3/users/67e9a0cec46f44a9b7cc709ef47fd664/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:34.791246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50ff4c-22f0-40c9-82fd-11873609b77e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.791772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50ff4c-22f0-40c9-82fd-11873609b77e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.792447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50ff4c-22f0-40c9-82fd-11873609b77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.831201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9b50ff4c-22f0-40c9-82fd-11873609b77e None None] MFA Rules not processed for user `67e9a0cec46f44a9b7cc709ef47fd664`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:34.832634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9b50ff4c-22f0-40c9-82fd-11873609b77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.942170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1094/2174] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:34.954321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2dbdd-425d-49f0-b04f-b78c23df29e9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.956165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-74e2dbdd-425d-49f0-b04f-b78c23df29e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.994728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:35.033299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2dbdd-425d-49f0-b04f-b78c23df29e9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.038604 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c24a04-316a-4472-9cfc-2426f7464571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.039199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c24a04-316a-4472-9cfc-2426f7464571 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.039473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c24a04-316a-4472-9cfc-2426f7464571 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.039741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c24a04-316a-4472-9cfc-2426f7464571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.055752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c24a04-316a-4472-9cfc-2426f7464571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.057606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c24a04-316a-4472-9cfc-2426f7464571 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.078320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1081/2175] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:35.117060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50ff4c-22f0-40c9-82fd-11873609b77e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.119194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9b50ff4c-22f0-40c9-82fd-11873609b77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.148291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c24a04-316a-4472-9cfc-2426f7464571 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.150475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a0c24a04-316a-4472-9cfc-2426f7464571 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.163452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c24a04-316a-4472-9cfc-2426f7464571 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.167423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a61ee-5744-4f58-afa1-6a8d73e008e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.168038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a61ee-5744-4f58-afa1-6a8d73e008e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.168322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a61ee-5744-4f58-afa1-6a8d73e008e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.168574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a61ee-5744-4f58-afa1-6a8d73e008e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.171493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ac8a61ee-5744-4f58-afa1-6a8d73e008e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.193272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:35.233124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50ff4c-22f0-40c9-82fd-11873609b77e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.238780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.245030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.245030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.245030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] PATH_INFO: `/v3/projects/630d4584c0ec4311b247a57ad19a5c75` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.246213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] RBAC: Authorizing `identity:delete_project(project_id=630d4584c0ec4311b247a57ad19a5c75)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.248759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.272886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a61ee-5744-4f58-afa1-6a8d73e008e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.274468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a61ee-5744-4f58-afa1-6a8d73e008e2 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.286401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1082/2176] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:35.338931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630d4584c0ec4311b247a57ad19a5c75', 'request_id': 'req-15c0f1d5-02dc-4745-bb0e-08c6e792651d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.348698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630d4584c0ec4311b247a57ad19a5c75', 'request_id': 'req-15c0f1d5-02dc-4745-bb0e-08c6e792651d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.359557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630d4584c0ec4311b247a57ad19a5c75', 'request_id': 'req-15c0f1d5-02dc-4745-bb0e-08c6e792651d'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.372794 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] The token cache is being invalidate because project 630d4584c0ec4311b247a57ad19a5c75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:35.373723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.374393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-15c0f1d5-02dc-4745-bb0e-08c6e792651d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.376778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1095/2177] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:35 2022] DELETE /identity/v3/projects/630d4584c0ec4311b247a57ad19a5c75 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:35.382680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33627ace-6d8e-42b1-bd7a-a575a10c6341 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.382912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33627ace-6d8e-42b1-bd7a-a575a10c6341 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.383179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33627ace-6d8e-42b1-bd7a-a575a10c6341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.405851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-33627ace-6d8e-42b1-bd7a-a575a10c6341 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:35.407298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-33627ace-6d8e-42b1-bd7a-a575a10c6341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.475321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1083/2178] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:35.484400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0953f-febc-4319-b7b4-b14e5ff96617 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.484687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0953f-febc-4319-b7b4-b14e5ff96617 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.484961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0953f-febc-4319-b7b4-b14e5ff96617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.517231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b0a0953f-febc-4319-b7b4-b14e5ff96617 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:35.518510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b0a0953f-febc-4319-b7b4-b14e5ff96617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.528826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1096/2179] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:35.533439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33627ace-6d8e-42b1-bd7a-a575a10c6341 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.545066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33627ace-6d8e-42b1-bd7a-a575a10c6341 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.550591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70d5cc-5343-468b-8e3b-1fac75deb669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.551617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70d5cc-5343-468b-8e3b-1fac75deb669 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.551924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70d5cc-5343-468b-8e3b-1fac75deb669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.552199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70d5cc-5343-468b-8e3b-1fac75deb669 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.552927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70d5cc-5343-468b-8e3b-1fac75deb669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:35.553267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70d5cc-5343-468b-8e3b-1fac75deb669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.554851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70d5cc-5343-468b-8e3b-1fac75deb669 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.581138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1084/2180] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:35.585325 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0953f-febc-4319-b7b4-b14e5ff96617 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.598311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0953f-febc-4319-b7b4-b14e5ff96617 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.605707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa599ec-8bf7-4684-8d7f-6415b4636122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.606288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa599ec-8bf7-4684-8d7f-6415b4636122 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.606565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa599ec-8bf7-4684-8d7f-6415b4636122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.607017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa599ec-8bf7-4684-8d7f-6415b4636122 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.608318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa599ec-8bf7-4684-8d7f-6415b4636122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.610373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa599ec-8bf7-4684-8d7f-6415b4636122 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.645110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1097/2181] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:35 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:35.648910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70d5cc-5343-468b-8e3b-1fac75deb669 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.662501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70d5cc-5343-468b-8e3b-1fac75deb669 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.734439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bde079-2019-4f4d-be5b-93c724390f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.735612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde079-2019-4f4d-be5b-93c724390f1b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.735612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde079-2019-4f4d-be5b-93c724390f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.735612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde079-2019-4f4d-be5b-93c724390f1b admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.736419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bde079-2019-4f4d-be5b-93c724390f1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.738485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bde079-2019-4f4d-be5b-93c724390f1b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.751229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:35.770828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1085/2182] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:35 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:35.775140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa599ec-8bf7-4684-8d7f-6415b4636122 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.786432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa599ec-8bf7-4684-8d7f-6415b4636122 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.790993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668ec32-a16f-4b4e-b01b-90512c8296bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.793530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668ec32-a16f-4b4e-b01b-90512c8296bf admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.793530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668ec32-a16f-4b4e-b01b-90512c8296bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.793530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668ec32-a16f-4b4e-b01b-90512c8296bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.793530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668ec32-a16f-4b4e-b01b-90512c8296bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.795167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668ec32-a16f-4b4e-b01b-90512c8296bf admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.809133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1098/2183] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:35.811627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bde079-2019-4f4d-be5b-93c724390f1b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.821912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bde079-2019-4f4d-be5b-93c724390f1b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.827521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70689910-13bb-42b6-9333-44bcfed45df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.827800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70689910-13bb-42b6-9333-44bcfed45df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.828073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70689910-13bb-42b6-9333-44bcfed45df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.828342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70689910-13bb-42b6-9333-44bcfed45df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.829415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70689910-13bb-42b6-9333-44bcfed45df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.830782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70689910-13bb-42b6-9333-44bcfed45df9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.841714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1086/2184] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:35.845993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668ec32-a16f-4b4e-b01b-90512c8296bf admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.856511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668ec32-a16f-4b4e-b01b-90512c8296bf admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.863296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7cab7-4323-4c19-aa9b-22c791c46290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.863958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7cab7-4323-4c19-aa9b-22c791c46290 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.864239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7cab7-4323-4c19-aa9b-22c791c46290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.864635 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7cab7-4323-4c19-aa9b-22c791c46290 admin admin] PATH_INFO: `/v3/projects/8a8b8077287845c1815a82a8fdbf51b6/users/a2c60f3426484bc0baa3107ae3568724/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.879357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70689910-13bb-42b6-9333-44bcfed45df9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.881310 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-70689910-13bb-42b6-9333-44bcfed45df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.883073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7cab7-4323-4c19-aa9b-22c791c46290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8b8077287845c1815a82a8fdbf51b6, user_id=a2c60f3426484bc0baa3107ae3568724, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.885868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7cab7-4323-4c19-aa9b-22c791c46290 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.898317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70689910-13bb-42b6-9333-44bcfed45df9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.902252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1099/2185] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:35 2022] PUT /identity/v3/projects/8a8b8077287845c1815a82a8fdbf51b6/users/a2c60f3426484bc0baa3107ae3568724/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:35.913865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aefc73-ffe0-4ffc-a569-6440d64c639e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.914253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aefc73-ffe0-4ffc-a569-6440d64c639e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.914593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aefc73-ffe0-4ffc-a569-6440d64c639e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.943292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-88aefc73-ffe0-4ffc-a569-6440d64c639e None None] MFA Rules not processed for user `a2c60f3426484bc0baa3107ae3568724`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:35.944498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-88aefc73-ffe0-4ffc-a569-6440d64c639e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.958074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.958696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.958970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.959237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] PATH_INFO: `/v3/users/7323455bb13e4cbb8c5b7c20975a3b59` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.960380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7323455bb13e4cbb8c5b7c20975a3b59)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.962352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.005871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:36.016131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7323455bb13e4cbb8c5b7c20975a3b59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.028534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7323455bb13e4cbb8c5b7c20975a3b59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.031633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1100/2186] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:36.032816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7323455bb13e4cbb8c5b7c20975a3b59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.034353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7323455bb13e4cbb8c5b7c20975a3b59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.036990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aefc73-ffe0-4ffc-a569-6440d64c639e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.038888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-88aefc73-ffe0-4ffc-a569-6440d64c639e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.051972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7323455bb13e4cbb8c5b7c20975a3b59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.070018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1087/2187] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:35 2022] DELETE /identity/v3/users/7323455bb13e4cbb8c5b7c20975a3b59 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:36.107862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aefc73-ffe0-4ffc-a569-6440d64c639e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.164196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55fa05-f952-428b-8152-71d977e87ad7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.164812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55fa05-f952-428b-8152-71d977e87ad7 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.165105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55fa05-f952-428b-8152-71d977e87ad7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.165389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55fa05-f952-428b-8152-71d977e87ad7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.167235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9b55fa05-f952-428b-8152-71d977e87ad7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.257397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55fa05-f952-428b-8152-71d977e87ad7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.259176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55fa05-f952-428b-8152-71d977e87ad7 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.275631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.277001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.283225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1101/2188] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:36.357231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:36.362357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9fcce-5367-462c-b9f4-d3249aedf754 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.369610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.370568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.371135 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.371458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] PATH_INFO: `/v3/projects/d2ca9cf5f6244ad5bb17bcfd1a4c4573` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.373614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ca9cf5f6244ad5bb17bcfd1a4c4573)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.375539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.431290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ca9cf5f6244ad5bb17bcfd1a4c4573', 'request_id': 'req-6ceb104d-bed6-4d22-95d4-a83668156327'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.438186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ca9cf5f6244ad5bb17bcfd1a4c4573', 'request_id': 'req-6ceb104d-bed6-4d22-95d4-a83668156327'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.448186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ca9cf5f6244ad5bb17bcfd1a4c4573', 'request_id': 'req-6ceb104d-bed6-4d22-95d4-a83668156327'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.459767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] The token cache is being invalidate because project d2ca9cf5f6244ad5bb17bcfd1a4c4573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:36.460058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.460700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.462378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1088/2189] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:36 2022] DELETE /identity/v3/projects/d2ca9cf5f6244ad5bb17bcfd1a4c4573 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:36.467147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55fa05-f952-428b-8152-71d977e87ad7 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.478277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55fa05-f952-428b-8152-71d977e87ad7 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.540054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.540721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.540973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.541247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] PATH_INFO: `/v3/users/e92cc2e2d0b64a7e88524e667bb90778` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.542602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e92cc2e2d0b64a7e88524e667bb90778)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.545919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.551689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:36.599808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e92cc2e2d0b64a7e88524e667bb90778'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.609652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e92cc2e2d0b64a7e88524e667bb90778'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.617611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e92cc2e2d0b64a7e88524e667bb90778'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.618323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e92cc2e2d0b64a7e88524e667bb90778'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.630912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a59ff3a3-46f2-4595-a7ac-6a246f16443f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e92cc2e2d0b64a7e88524e667bb90778'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.636764 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1102/2190] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:36 2022] DELETE /identity/v3/users/e92cc2e2d0b64a7e88524e667bb90778 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:36.823808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.826408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.846698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb104d-bed6-4d22-95d4-a83668156327 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.922961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.923594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.923872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.924144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] PATH_INFO: `/v3/projects/90f92a8ecdcf4fa38d9108c6cc5d3c48` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.925296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f92a8ecdcf4fa38d9108c6cc5d3c48)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.927184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.984594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f92a8ecdcf4fa38d9108c6cc5d3c48', 'request_id': 'req-136b983d-d0d5-478d-ac49-bcbe3ec0880c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.991152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f92a8ecdcf4fa38d9108c6cc5d3c48', 'request_id': 'req-136b983d-d0d5-478d-ac49-bcbe3ec0880c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:37.000313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f92a8ecdcf4fa38d9108c6cc5d3c48', 'request_id': 'req-136b983d-d0d5-478d-ac49-bcbe3ec0880c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:37.016209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] The token cache is being invalidate because project 90f92a8ecdcf4fa38d9108c6cc5d3c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:37.016498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:37.017259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-136b983d-d0d5-478d-ac49-bcbe3ec0880c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:37.018762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1089/2191] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:36 2022] DELETE /identity/v3/projects/90f92a8ecdcf4fa38d9108c6cc5d3c48 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:37.024347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f4275-b0cf-4a27-96d6-6e29a43e57c3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.024629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f4275-b0cf-4a27-96d6-6e29a43e57c3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.024902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f4275-b0cf-4a27-96d6-6e29a43e57c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.046722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-845f4275-b0cf-4a27-96d6-6e29a43e57c3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:37.047926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-845f4275-b0cf-4a27-96d6-6e29a43e57c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:37.125110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1103/2192] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:37.129782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bef8c0-566e-4e88-805f-a9e8aa156780 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.130271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bef8c0-566e-4e88-805f-a9e8aa156780 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.130369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bef8c0-566e-4e88-805f-a9e8aa156780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.151077 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-06bef8c0-566e-4e88-805f-a9e8aa156780 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:37.152276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-06bef8c0-566e-4e88-805f-a9e8aa156780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:37.161703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1090/2193] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:37.166073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f4275-b0cf-4a27-96d6-6e29a43e57c3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.177747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f4275-b0cf-4a27-96d6-6e29a43e57c3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.183031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701eff8-bd5c-4fbc-929a-ac308afdc61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.183754 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701eff8-bd5c-4fbc-929a-ac308afdc61c admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.184159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701eff8-bd5c-4fbc-929a-ac308afdc61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.184437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701eff8-bd5c-4fbc-929a-ac308afdc61c admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.185960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701eff8-bd5c-4fbc-929a-ac308afdc61c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:37.186404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701eff8-bd5c-4fbc-929a-ac308afdc61c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.189513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701eff8-bd5c-4fbc-929a-ac308afdc61c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.202747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1104/2194] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:37.207504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bef8c0-566e-4e88-805f-a9e8aa156780 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.229493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bef8c0-566e-4e88-805f-a9e8aa156780 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.235508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee448bd-25f8-4e85-8726-d06b8db7c024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.236119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee448bd-25f8-4e85-8726-d06b8db7c024 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.236394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee448bd-25f8-4e85-8726-d06b8db7c024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.236661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee448bd-25f8-4e85-8726-d06b8db7c024 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.237344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee448bd-25f8-4e85-8726-d06b8db7c024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.239252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee448bd-25f8-4e85-8726-d06b8db7c024 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.250788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.271344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1091/2195] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:37.275546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701eff8-bd5c-4fbc-929a-ac308afdc61c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.286367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701eff8-bd5c-4fbc-929a-ac308afdc61c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.353396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4964f8d-921c-4df8-ae58-17c74ccff445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.354081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4964f8d-921c-4df8-ae58-17c74ccff445 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.354432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4964f8d-921c-4df8-ae58-17c74ccff445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.355001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4964f8d-921c-4df8-ae58-17c74ccff445 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.355654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4964f8d-921c-4df8-ae58-17c74ccff445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.359323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4964f8d-921c-4df8-ae58-17c74ccff445 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.385263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1105/2196] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:37.388623 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee448bd-25f8-4e85-8726-d06b8db7c024 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.400048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee448bd-25f8-4e85-8726-d06b8db7c024 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.405988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce378555-b8fa-486b-954c-0dc939bf76c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.406723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce378555-b8fa-486b-954c-0dc939bf76c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.406957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce378555-b8fa-486b-954c-0dc939bf76c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.407233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce378555-b8fa-486b-954c-0dc939bf76c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.408143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce378555-b8fa-486b-954c-0dc939bf76c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.410079 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce378555-b8fa-486b-954c-0dc939bf76c2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.424770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1092/2197] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:37.426104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4964f8d-921c-4df8-ae58-17c74ccff445 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.431648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.438715 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4964f8d-921c-4df8-ae58-17c74ccff445 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.445894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120a576-d70f-4357-8ad9-4b9b53157d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.446724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120a576-d70f-4357-8ad9-4b9b53157d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.446932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120a576-d70f-4357-8ad9-4b9b53157d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.447204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120a576-d70f-4357-8ad9-4b9b53157d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.447848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120a576-d70f-4357-8ad9-4b9b53157d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.449751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120a576-d70f-4357-8ad9-4b9b53157d4b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.462124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1106/2198] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:37.466198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce378555-b8fa-486b-954c-0dc939bf76c2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.480782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce378555-b8fa-486b-954c-0dc939bf76c2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.487417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a51c30-a30e-4ab6-8a13-892ae0998b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.488155 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a51c30-a30e-4ab6-8a13-892ae0998b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.488501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a51c30-a30e-4ab6-8a13-892ae0998b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.488844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a51c30-a30e-4ab6-8a13-892ae0998b9d admin admin] PATH_INFO: `/v3/projects/1df4efa5824142119f6355beac8ffa21/users/b556b5910f614548a766fff3f115460c/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.508211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a51c30-a30e-4ab6-8a13-892ae0998b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df4efa5824142119f6355beac8ffa21, user_id=b556b5910f614548a766fff3f115460c, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.511555 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a51c30-a30e-4ab6-8a13-892ae0998b9d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.527497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1093/2199] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:37 2022] PUT /identity/v3/projects/1df4efa5824142119f6355beac8ffa21/users/b556b5910f614548a766fff3f115460c/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:37.538180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882f3ab-a58f-4c33-a8c9-9984968c38b3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.538469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882f3ab-a58f-4c33-a8c9-9984968c38b3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.538768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882f3ab-a58f-4c33-a8c9-9984968c38b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.573975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-6882f3ab-a58f-4c33-a8c9-9984968c38b3 None None] MFA Rules not processed for user `b556b5910f614548a766fff3f115460c`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:37.575492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6882f3ab-a58f-4c33-a8c9-9984968c38b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:37.648555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1107/2200] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:37.652902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882f3ab-a58f-4c33-a8c9-9984968c38b3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.653065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bca79-d037-45ba-b7ee-1d63453d9741 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.653338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bca79-d037-45ba-b7ee-1d63453d9741 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.653611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bca79-d037-45ba-b7ee-1d63453d9741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.674991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8b3bca79-d037-45ba-b7ee-1d63453d9741 None None] MFA Rules not processed for user `a2c60f3426484bc0baa3107ae3568724`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:37.676203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8b3bca79-d037-45ba-b7ee-1d63453d9741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:37.721628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882f3ab-a58f-4c33-a8c9-9984968c38b3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.726571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27973c-0888-49f2-8739-39a239433148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.727102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27973c-0888-49f2-8739-39a239433148 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.727372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27973c-0888-49f2-8739-39a239433148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.727636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27973c-0888-49f2-8739-39a239433148 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.728260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27973c-0888-49f2-8739-39a239433148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.730493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27973c-0888-49f2-8739-39a239433148 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.739643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1094/2201] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:37.777121 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1108/2202] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:37.778430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bca79-d037-45ba-b7ee-1d63453d9741 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.797255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bca79-d037-45ba-b7ee-1d63453d9741 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.861234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6fd1e-69ab-4fc6-ac4c-3bdf1f0d78f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.862412 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6fd1e-69ab-4fc6-ac4c-3bdf1f0d78f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.862913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6fd1e-69ab-4fc6-ac4c-3bdf1f0d78f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.863402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6fd1e-69ab-4fc6-ac4c-3bdf1f0d78f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.864291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6fd1e-69ab-4fc6-ac4c-3bdf1f0d78f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.866465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6fd1e-69ab-4fc6-ac4c-3bdf1f0d78f5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.890491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1095/2203] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:37.894489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27973c-0888-49f2-8739-39a239433148 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.905902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27973c-0888-49f2-8739-39a239433148 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.910652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24972-e9a4-419d-acde-52a308943098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.911251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24972-e9a4-419d-acde-52a308943098 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.911532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24972-e9a4-419d-acde-52a308943098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.911804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24972-e9a4-419d-acde-52a308943098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.912444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da24972-e9a4-419d-acde-52a308943098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.914071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da24972-e9a4-419d-acde-52a308943098 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.925380 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1109/2204] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:37.929766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6fd1e-69ab-4fc6-ac4c-3bdf1f0d78f5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.941146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6fd1e-69ab-4fc6-ac4c-3bdf1f0d78f5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.946067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7207f-988b-473d-a470-64a2cc518837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.946788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7207f-988b-473d-a470-64a2cc518837 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.947136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7207f-988b-473d-a470-64a2cc518837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.947479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7207f-988b-473d-a470-64a2cc518837 admin admin] PATH_INFO: `/v3/projects/64c4fa869b124a01bc217251fe7389d9/users/17619f0c01f24fb9af9b3766a0d8485f/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.963760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7207f-988b-473d-a470-64a2cc518837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c4fa869b124a01bc217251fe7389d9, user_id=17619f0c01f24fb9af9b3766a0d8485f, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.966506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7207f-988b-473d-a470-64a2cc518837 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.980230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1096/2205] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:37 2022] PUT /identity/v3/projects/64c4fa869b124a01bc217251fe7389d9/users/17619f0c01f24fb9af9b3766a0d8485f/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:37.985039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24972-e9a4-419d-acde-52a308943098 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.998835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24972-e9a4-419d-acde-52a308943098 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:38.061552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:38.080879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea4034-55da-432e-83ce-e467ebe869bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:38.081534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea4034-55da-432e-83ce-e467ebe869bd admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.081813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea4034-55da-432e-83ce-e467ebe869bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.082080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea4034-55da-432e-83ce-e467ebe869bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.082724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea4034-55da-432e-83ce-e467ebe869bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:38.084319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea4034-55da-432e-83ce-e467ebe869bd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:38.095459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1110/2206] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:38.099641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7207f-988b-473d-a470-64a2cc518837 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:38.111256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7207f-988b-473d-a470-64a2cc518837 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:38.116071 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd3d87-dba9-46bd-ade9-ce98d5a3adfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:38.116726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd3d87-dba9-46bd-ade9-ce98d5a3adfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.117007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd3d87-dba9-46bd-ade9-ce98d5a3adfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.117337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd3d87-dba9-46bd-ade9-ce98d5a3adfb admin admin] PATH_INFO: `/v3/projects/64c4fa869b124a01bc217251fe7389d9/users/17619f0c01f24fb9af9b3766a0d8485f/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.119321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd3d87-dba9-46bd-ade9-ce98d5a3adfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c4fa869b124a01bc217251fe7389d9, user_id=17619f0c01f24fb9af9b3766a0d8485f, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:38.122048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd3d87-dba9-46bd-ade9-ce98d5a3adfb admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:38.135942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1097/2207] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:38 2022] PUT /identity/v3/projects/64c4fa869b124a01bc217251fe7389d9/users/17619f0c01f24fb9af9b3766a0d8485f/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:38.147864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187bc44-95b4-4188-b3fc-a8b00f37781a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.148240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187bc44-95b4-4188-b3fc-a8b00f37781a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.148520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187bc44-95b4-4188-b3fc-a8b00f37781a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.182237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b187bc44-95b4-4188-b3fc-a8b00f37781a None None] MFA Rules not processed for user `17619f0c01f24fb9af9b3766a0d8485f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:38.183573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b187bc44-95b4-4188-b3fc-a8b00f37781a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:38.297477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1111/2208] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:38.313051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd3d87-dba9-46bd-ade9-ce98d5a3adfb admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:38.315254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e6dd3d87-dba9-46bd-ade9-ce98d5a3adfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:38.332665 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:38.380664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187bc44-95b4-4188-b3fc-a8b00f37781a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:38.382117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b187bc44-95b4-4188-b3fc-a8b00f37781a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:38.425121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd3d87-dba9-46bd-ade9-ce98d5a3adfb admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:38.429645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4be8c-5bd1-47cc-8bba-c34920315bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:38.430185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4be8c-5bd1-47cc-8bba-c34920315bdd service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.430475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4be8c-5bd1-47cc-8bba-c34920315bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.431261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4be8c-5bd1-47cc-8bba-c34920315bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.459291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187bc44-95b4-4188-b3fc-a8b00f37781a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:38.464640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:38.465391 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.465585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.465863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] PATH_INFO: `/v3/users/a8b7acb2dcc6496ca322c00793443d0f` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.467698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b7acb2dcc6496ca322c00793443d0f)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:38.470430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:38.499720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4be8c-5bd1-47cc-8bba-c34920315bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:38.501364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4be8c-5bd1-47cc-8bba-c34920315bdd service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:38.512378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1098/2209] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:38.534439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b7acb2dcc6496ca322c00793443d0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:38.543216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b7acb2dcc6496ca322c00793443d0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:38.553388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b7acb2dcc6496ca322c00793443d0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:38.554050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b7acb2dcc6496ca322c00793443d0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:38.571833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:38.573991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b7acb2dcc6496ca322c00793443d0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:38.588590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1112/2210] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:38 2022] DELETE /identity/v3/users/a8b7acb2dcc6496ca322c00793443d0f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:38.594028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480d6dc-c0a5-431c-8e64-99701e6fa436 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.594447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480d6dc-c0a5-431c-8e64-99701e6fa436 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.594797 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480d6dc-c0a5-431c-8e64-99701e6fa436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.620463 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9480d6dc-c0a5-431c-8e64-99701e6fa436 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:38.622572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9480d6dc-c0a5-431c-8e64-99701e6fa436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:38.717944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1099/2211] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:38.729191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:38.730693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:38.809451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff155aa-64e9-4e76-976e-015f8231fe4f admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:38.815351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a3a4d-0b3c-4272-bb94-26536a97ae20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:38.816274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a3a4d-0b3c-4272-bb94-26536a97ae20 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.816623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a3a4d-0b3c-4272-bb94-26536a97ae20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.816990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a3a4d-0b3c-4272-bb94-26536a97ae20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.831440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a3a4d-0b3c-4272-bb94-26536a97ae20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:38.832898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a3a4d-0b3c-4272-bb94-26536a97ae20 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:38.843206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1113/2212] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:39.142396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480d6dc-c0a5-431c-8e64-99701e6fa436 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:39.143961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9480d6dc-c0a5-431c-8e64-99701e6fa436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:39.156700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480d6dc-c0a5-431c-8e64-99701e6fa436 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:39.161382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:39.162013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.162290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.162562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] PATH_INFO: `/v3/projects/156335eaf6384d568d74957a1b4e4f4b` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.163734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] RBAC: Authorizing `identity:delete_project(project_id=156335eaf6384d568d74957a1b4e4f4b)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:39.165877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:39.214435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156335eaf6384d568d74957a1b4e4f4b', 'request_id': 'req-e159cfc6-aa37-446a-b63a-bfb90a83493f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.221255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156335eaf6384d568d74957a1b4e4f4b', 'request_id': 'req-e159cfc6-aa37-446a-b63a-bfb90a83493f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.228638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156335eaf6384d568d74957a1b4e4f4b', 'request_id': 'req-e159cfc6-aa37-446a-b63a-bfb90a83493f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.230026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:39.241683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] The token cache is being invalidate because project 156335eaf6384d568d74957a1b4e4f4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:39.242006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.242896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.244602 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1100/2213] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:39 2022] DELETE /identity/v3/projects/156335eaf6384d568d74957a1b4e4f4b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:39.249170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a3a4d-0b3c-4272-bb94-26536a97ae20 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:39.259046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a3a4d-0b3c-4272-bb94-26536a97ae20 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:39.321995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:39.322675 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.322886 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.323191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] PATH_INFO: `/v3/users/9ee2031646304d1293cbb93b25dc4161` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.324360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee2031646304d1293cbb93b25dc4161)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:39.327213 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:39.374903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee2031646304d1293cbb93b25dc4161'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.383039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee2031646304d1293cbb93b25dc4161'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.391396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee2031646304d1293cbb93b25dc4161'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.392093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee2031646304d1293cbb93b25dc4161'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.404612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee2031646304d1293cbb93b25dc4161'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.410397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1114/2214] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:39 2022] DELETE /identity/v3/users/9ee2031646304d1293cbb93b25dc4161 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:39.699166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:39.700696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:39.718164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159cfc6-aa37-446a-b63a-bfb90a83493f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:39.782274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:39.782821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.783101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.783371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] PATH_INFO: `/v3/projects/9fc0f43ed2e54de0a0cd62394b598409` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.784480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc0f43ed2e54de0a0cd62394b598409)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:39.786395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:39.848888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc0f43ed2e54de0a0cd62394b598409', 'request_id': 'req-5fed2462-7524-4ccb-acf4-85c792f9450e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.855735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc0f43ed2e54de0a0cd62394b598409', 'request_id': 'req-5fed2462-7524-4ccb-acf4-85c792f9450e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.869583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc0f43ed2e54de0a0cd62394b598409', 'request_id': 'req-5fed2462-7524-4ccb-acf4-85c792f9450e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.883132 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] The token cache is being invalidate because project 9fc0f43ed2e54de0a0cd62394b598409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:39.883390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.883943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:39.885727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1101/2215] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:39 2022] DELETE /identity/v3/projects/9fc0f43ed2e54de0a0cd62394b598409 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:39.889953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:39.891483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:39.905823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234a722-1119-4024-b9e4-ef3802f3738b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:39.918478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:39.973273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:39.973933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.974212 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.974482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] PATH_INFO: `/v3/users/67e9a0cec46f44a9b7cc709ef47fd664` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.975628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67e9a0cec46f44a9b7cc709ef47fd664)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:39.977608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:40.037825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67e9a0cec46f44a9b7cc709ef47fd664'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.045725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67e9a0cec46f44a9b7cc709ef47fd664'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.054676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67e9a0cec46f44a9b7cc709ef47fd664'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.055625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67e9a0cec46f44a9b7cc709ef47fd664'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.071086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7f0e92db-516c-47a5-afea-89271f2d8165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67e9a0cec46f44a9b7cc709ef47fd664'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.083496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1115/2216] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:39 2022] DELETE /identity/v3/users/67e9a0cec46f44a9b7cc709ef47fd664 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:40.468317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.469927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:40.495558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed2462-7524-4ccb-acf4-85c792f9450e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:40.567063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:40.567859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.568266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.568626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] PATH_INFO: `/v3/projects/3cf1c667947e45cdb95ccad299a6ecc3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.569954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf1c667947e45cdb95ccad299a6ecc3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:40.571927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:40.599633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:40.639683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf1c667947e45cdb95ccad299a6ecc3', 'request_id': 'req-319bc903-9471-4b3d-85b8-a6bf8e759228'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.648438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf1c667947e45cdb95ccad299a6ecc3', 'request_id': 'req-319bc903-9471-4b3d-85b8-a6bf8e759228'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.661411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf1c667947e45cdb95ccad299a6ecc3', 'request_id': 'req-319bc903-9471-4b3d-85b8-a6bf8e759228'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.676426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] The token cache is being invalidate because project 3cf1c667947e45cdb95ccad299a6ecc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:40.677156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.677895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-319bc903-9471-4b3d-85b8-a6bf8e759228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:40.679487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1102/2217] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:40 2022] DELETE /identity/v3/projects/3cf1c667947e45cdb95ccad299a6ecc3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:40.686772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be0e58-3f85-4a2f-880a-c49cd0531466 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.687266 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be0e58-3f85-4a2f-880a-c49cd0531466 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.687358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be0e58-3f85-4a2f-880a-c49cd0531466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.707503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b8be0e58-3f85-4a2f-880a-c49cd0531466 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:40.708839 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b8be0e58-3f85-4a2f-880a-c49cd0531466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:40.813619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1116/2218] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:40.821953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7da31-b7ba-4149-a43a-6215262136f3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.822176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7da31-b7ba-4149-a43a-6215262136f3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.822505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7da31-b7ba-4149-a43a-6215262136f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.846121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-45b7da31-b7ba-4149-a43a-6215262136f3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:40.847436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-45b7da31-b7ba-4149-a43a-6215262136f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:40.862941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1103/2219] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:40.863885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be0e58-3f85-4a2f-880a-c49cd0531466 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.874946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be0e58-3f85-4a2f-880a-c49cd0531466 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:40.881004 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5000084-7661-4b8a-8af3-5ed440f39cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:40.881646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5000084-7661-4b8a-8af3-5ed440f39cde admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.881954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5000084-7661-4b8a-8af3-5ed440f39cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.882230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5000084-7661-4b8a-8af3-5ed440f39cde admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.882878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5000084-7661-4b8a-8af3-5ed440f39cde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:40.883290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5000084-7661-4b8a-8af3-5ed440f39cde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:40.884807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5000084-7661-4b8a-8af3-5ed440f39cde admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:40.897463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1117/2220] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:40.901576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7da31-b7ba-4149-a43a-6215262136f3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.911792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7da31-b7ba-4149-a43a-6215262136f3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:40.916981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ca3b7-92fa-4143-9e23-9e0783ec7d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:40.917638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ca3b7-92fa-4143-9e23-9e0783ec7d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.917912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ca3b7-92fa-4143-9e23-9e0783ec7d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.918180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ca3b7-92fa-4143-9e23-9e0783ec7d7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.918880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ca3b7-92fa-4143-9e23-9e0783ec7d7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:40.921964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ca3b7-92fa-4143-9e23-9e0783ec7d7b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:40.955435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1104/2221] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:40 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:40.959248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5000084-7661-4b8a-8af3-5ed440f39cde admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.972457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5000084-7661-4b8a-8af3-5ed440f39cde admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:40.997321 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:41.054292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dee97f-132f-4dd8-87b9-e44ae62f2b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.054826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dee97f-132f-4dd8-87b9-e44ae62f2b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.055172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dee97f-132f-4dd8-87b9-e44ae62f2b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.055382 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dee97f-132f-4dd8-87b9-e44ae62f2b4c admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.056182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dee97f-132f-4dd8-87b9-e44ae62f2b4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.058762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dee97f-132f-4dd8-87b9-e44ae62f2b4c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.090093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1118/2222] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:40 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:41.091434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ca3b7-92fa-4143-9e23-9e0783ec7d7b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.102735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ca3b7-92fa-4143-9e23-9e0783ec7d7b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.108667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59245a2-ae15-453a-a39c-6aa4837ab5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.109317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59245a2-ae15-453a-a39c-6aa4837ab5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.109603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59245a2-ae15-453a-a39c-6aa4837ab5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.109880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59245a2-ae15-453a-a39c-6aa4837ab5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.110527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59245a2-ae15-453a-a39c-6aa4837ab5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.112159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59245a2-ae15-453a-a39c-6aa4837ab5d4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.125107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1105/2223] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:41.127335 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dee97f-132f-4dd8-87b9-e44ae62f2b4c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.137502 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dee97f-132f-4dd8-87b9-e44ae62f2b4c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.142226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044ff5-140f-4232-915f-b8e46f928e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.143448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044ff5-140f-4232-915f-b8e46f928e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.143737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044ff5-140f-4232-915f-b8e46f928e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.144012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044ff5-140f-4232-915f-b8e46f928e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.144662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98044ff5-140f-4232-915f-b8e46f928e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.146292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98044ff5-140f-4232-915f-b8e46f928e64 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.160731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1119/2224] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:41.161729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59245a2-ae15-453a-a39c-6aa4837ab5d4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.174993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59245a2-ae15-453a-a39c-6aa4837ab5d4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.181246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c435d-b48a-4083-a0db-b35adb66c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.181993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c435d-b48a-4083-a0db-b35adb66c633 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.182374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c435d-b48a-4083-a0db-b35adb66c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.182850 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c435d-b48a-4083-a0db-b35adb66c633 admin admin] PATH_INFO: `/v3/projects/1971fa3358044f5ba43c88a3abdabb5f/users/e84a45debfc246c28ccf6d4be5129232/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.204722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044ff5-140f-4232-915f-b8e46f928e64 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.207088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-98044ff5-140f-4232-915f-b8e46f928e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:41.207411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c435d-b48a-4083-a0db-b35adb66c633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1971fa3358044f5ba43c88a3abdabb5f, user_id=e84a45debfc246c28ccf6d4be5129232, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.210860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c435d-b48a-4083-a0db-b35adb66c633 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.222282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044ff5-140f-4232-915f-b8e46f928e64 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.230595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1106/2225] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:41 2022] PUT /identity/v3/projects/1971fa3358044f5ba43c88a3abdabb5f/users/e84a45debfc246c28ccf6d4be5129232/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:41.257756 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce168a-07a9-4a6d-879b-6f9aa3d5f9c1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.258322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce168a-07a9-4a6d-879b-6f9aa3d5f9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.259007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce168a-07a9-4a6d-879b-6f9aa3d5f9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.299425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7cce168a-07a9-4a6d-879b-6f9aa3d5f9c1 None None] MFA Rules not processed for user `e84a45debfc246c28ccf6d4be5129232`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:41.300655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7cce168a-07a9-4a6d-879b-6f9aa3d5f9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:41.304560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.305568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.306129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.306458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] PATH_INFO: `/v3/users/b556b5910f614548a766fff3f115460c` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.308279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b556b5910f614548a766fff3f115460c)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.310903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.383759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b556b5910f614548a766fff3f115460c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:41.388078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1107/2226] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:41.393354 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce168a-07a9-4a6d-879b-6f9aa3d5f9c1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.406208 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b556b5910f614548a766fff3f115460c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:41.411514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce168a-07a9-4a6d-879b-6f9aa3d5f9c1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.416528 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b556b5910f614548a766fff3f115460c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:41.417400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b556b5910f614548a766fff3f115460c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:41.418097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c89b73-51c8-46a0-a581-045c0ebb83a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.419510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c89b73-51c8-46a0-a581-045c0ebb83a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.420075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c89b73-51c8-46a0-a581-045c0ebb83a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.420612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c89b73-51c8-46a0-a581-045c0ebb83a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.421434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c89b73-51c8-46a0-a581-045c0ebb83a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.423880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c89b73-51c8-46a0-a581-045c0ebb83a5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.433625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-7e564b5f-72c7-4dfa-8475-80462748b4b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b556b5910f614548a766fff3f115460c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:41.443840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1120/2227] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:41 2022] DELETE /identity/v3/users/b556b5910f614548a766fff3f115460c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:41.448570 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04af4e3-183c-436a-bfce-2f063d93bbd4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.448897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04af4e3-183c-436a-bfce-2f063d93bbd4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.449278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04af4e3-183c-436a-bfce-2f063d93bbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.461006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1108/2228] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:41 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:41.465657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c89b73-51c8-46a0-a581-045c0ebb83a5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.468732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-50c89b73-51c8-46a0-a581-045c0ebb83a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:41.471636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b04af4e3-183c-436a-bfce-2f063d93bbd4 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:41.472925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b04af4e3-183c-436a-bfce-2f063d93bbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:41.482905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:41.490338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:41.545233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1121/2229] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:41.554344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c89b73-51c8-46a0-a581-045c0ebb83a5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.557730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04af4e3-183c-436a-bfce-2f063d93bbd4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.559777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b04af4e3-183c-436a-bfce-2f063d93bbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:41.559959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a6823-33c5-472c-ba05-6169e219f9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.561470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a6823-33c5-472c-ba05-6169e219f9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.561745 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a6823-33c5-472c-ba05-6169e219f9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.562011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a6823-33c5-472c-ba05-6169e219f9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.562643 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a6823-33c5-472c-ba05-6169e219f9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.564535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a6823-33c5-472c-ba05-6169e219f9c9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.589294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1109/2230] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:41 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:41.593659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a6823-33c5-472c-ba05-6169e219f9c9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.603705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a6823-33c5-472c-ba05-6169e219f9c9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.610527 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0240c22-b7b5-488f-a23c-a663f772d27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.611129 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0240c22-b7b5-488f-a23c-a663f772d27c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.611664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0240c22-b7b5-488f-a23c-a663f772d27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.611944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0240c22-b7b5-488f-a23c-a663f772d27c admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.612596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0240c22-b7b5-488f-a23c-a663f772d27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.614311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0240c22-b7b5-488f-a23c-a663f772d27c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.625993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1110/2231] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:41.630226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04af4e3-183c-436a-bfce-2f063d93bbd4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.630432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0240c22-b7b5-488f-a23c-a663f772d27c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.633872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98072d9-2328-42e5-80bb-0a419d47c62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.634472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98072d9-2328-42e5-80bb-0a419d47c62b service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.634902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98072d9-2328-42e5-80bb-0a419d47c62b service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.635170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98072d9-2328-42e5-80bb-0a419d47c62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.642895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0240c22-b7b5-488f-a23c-a663f772d27c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.648803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98072d9-2328-42e5-80bb-0a419d47c62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.649827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ddfed-f116-4573-9f8a-5b38fb7191c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.650686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ddfed-f116-4573-9f8a-5b38fb7191c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.650821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98072d9-2328-42e5-80bb-0a419d47c62b service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.651062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ddfed-f116-4573-9f8a-5b38fb7191c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.651427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ddfed-f116-4573-9f8a-5b38fb7191c5 admin admin] PATH_INFO: `/v3/projects/c0458d0f9bf2475fb03fcd1d71120ff5/users/48723ba440504645877fb41a4ee89400/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.663711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1122/2232] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:41.672063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ddfed-f116-4573-9f8a-5b38fb7191c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0458d0f9bf2475fb03fcd1d71120ff5, user_id=48723ba440504645877fb41a4ee89400, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.676822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ddfed-f116-4573-9f8a-5b38fb7191c5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.695600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1111/2233] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:41 2022] PUT /identity/v3/projects/c0458d0f9bf2475fb03fcd1d71120ff5/users/48723ba440504645877fb41a4ee89400/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:41.699945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98072d9-2328-42e5-80bb-0a419d47c62b service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.712077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98072d9-2328-42e5-80bb-0a419d47c62b service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.785235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5f635-6096-4f77-becf-df286aa0a26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.786508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5f635-6096-4f77-becf-df286aa0a26d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.787150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5f635-6096-4f77-becf-df286aa0a26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.787639 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5f635-6096-4f77-becf-df286aa0a26d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.789728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5f635-6096-4f77-becf-df286aa0a26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.791774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5f635-6096-4f77-becf-df286aa0a26d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.805118 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1123/2234] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:41.809325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ddfed-f116-4573-9f8a-5b38fb7191c5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.823397 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ddfed-f116-4573-9f8a-5b38fb7191c5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.836955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992280d-fca0-4769-9c62-db9c1e456b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.837738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992280d-fca0-4769-9c62-db9c1e456b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.838520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992280d-fca0-4769-9c62-db9c1e456b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.838520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992280d-fca0-4769-9c62-db9c1e456b85 admin admin] PATH_INFO: `/v3/projects/c0458d0f9bf2475fb03fcd1d71120ff5/users/48723ba440504645877fb41a4ee89400/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.842774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992280d-fca0-4769-9c62-db9c1e456b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0458d0f9bf2475fb03fcd1d71120ff5, user_id=48723ba440504645877fb41a4ee89400, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.848444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992280d-fca0-4769-9c62-db9c1e456b85 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.861738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1112/2235] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:41 2022] PUT /identity/v3/projects/c0458d0f9bf2475fb03fcd1d71120ff5/users/48723ba440504645877fb41a4ee89400/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:41.872489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b35b21-9c68-4237-96d7-e5e282a0f5db None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.872880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b35b21-9c68-4237-96d7-e5e282a0f5db None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.873310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b35b21-9c68-4237-96d7-e5e282a0f5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.902010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-58b35b21-9c68-4237-96d7-e5e282a0f5db None None] MFA Rules not processed for user `48723ba440504645877fb41a4ee89400`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:41.903219 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-58b35b21-9c68-4237-96d7-e5e282a0f5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:41.967741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:41.990658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1124/2236] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:42.002897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992280d-fca0-4769-9c62-db9c1e456b85 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:42.004806 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d992280d-fca0-4769-9c62-db9c1e456b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:42.067765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992280d-fca0-4769-9c62-db9c1e456b85 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:42.071909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc897c7-9adb-4a00-873e-b2a22bafca0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:42.072377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc897c7-9adb-4a00-873e-b2a22bafca0b service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:42.072659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc897c7-9adb-4a00-873e-b2a22bafca0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:42.073902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc897c7-9adb-4a00-873e-b2a22bafca0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:42.077415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fcc897c7-9adb-4a00-873e-b2a22bafca0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:42.146150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc897c7-9adb-4a00-873e-b2a22bafca0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:42.147588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc897c7-9adb-4a00-873e-b2a22bafca0b service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:42.172007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1113/2237] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:42.272051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b35b21-9c68-4237-96d7-e5e282a0f5db None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:42.273554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-58b35b21-9c68-4237-96d7-e5e282a0f5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:42.416627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b35b21-9c68-4237-96d7-e5e282a0f5db None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:42.426937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:42.427594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:42.427876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:42.428154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] PATH_INFO: `/v3/projects/1df4efa5824142119f6355beac8ffa21` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:42.429362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1df4efa5824142119f6355beac8ffa21)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:42.443558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:42.508899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1df4efa5824142119f6355beac8ffa21', 'request_id': 'req-38c66924-16cf-406e-a5bb-b75324f68475'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.516509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1df4efa5824142119f6355beac8ffa21', 'request_id': 'req-38c66924-16cf-406e-a5bb-b75324f68475'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.528643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1df4efa5824142119f6355beac8ffa21', 'request_id': 'req-38c66924-16cf-406e-a5bb-b75324f68475'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.545633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] The token cache is being invalidate because project 1df4efa5824142119f6355beac8ffa21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:42.546076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.546752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.548420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1125/2238] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:42 2022] DELETE /identity/v3/projects/1df4efa5824142119f6355beac8ffa21 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:42.553281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc897c7-9adb-4a00-873e-b2a22bafca0b service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:42.567484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc897c7-9adb-4a00-873e-b2a22bafca0b service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:42.572483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:42.669243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:42.670128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:42.670418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:42.670745 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] PATH_INFO: `/v3/users/17619f0c01f24fb9af9b3766a0d8485f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:42.671919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] RBAC: Authorizing `identity:delete_user(user_id=17619f0c01f24fb9af9b3766a0d8485f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:42.674357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:42.731868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17619f0c01f24fb9af9b3766a0d8485f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.742140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17619f0c01f24fb9af9b3766a0d8485f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.743277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17619f0c01f24fb9af9b3766a0d8485f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.743968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17619f0c01f24fb9af9b3766a0d8485f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.760145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-915f1ae5-ac9c-4739-900c-0285038ac13a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17619f0c01f24fb9af9b3766a0d8485f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.774399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1114/2239] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:42 2022] DELETE /identity/v3/users/17619f0c01f24fb9af9b3766a0d8485f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:43.323225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:43.324708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:43.359969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c66924-16cf-406e-a5bb-b75324f68475 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:43.430416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:43.431174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.431985 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.432423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] PATH_INFO: `/v3/projects/64c4fa869b124a01bc217251fe7389d9` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.434537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c4fa869b124a01bc217251fe7389d9)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:43.437525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:43.461959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:43.495777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c4fa869b124a01bc217251fe7389d9', 'request_id': 'req-d11d1b8c-660d-4729-b445-632d109e059f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.504659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c4fa869b124a01bc217251fe7389d9', 'request_id': 'req-d11d1b8c-660d-4729-b445-632d109e059f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.519095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c4fa869b124a01bc217251fe7389d9', 'request_id': 'req-d11d1b8c-660d-4729-b445-632d109e059f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.533531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] The token cache is being invalidate because project 64c4fa869b124a01bc217251fe7389d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:43.533878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.534551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d11d1b8c-660d-4729-b445-632d109e059f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.536366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1126/2240] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:43 2022] DELETE /identity/v3/projects/64c4fa869b124a01bc217251fe7389d9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:43.542131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9da8-1095-46cd-a977-6eab1afab551 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.542524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9da8-1095-46cd-a977-6eab1afab551 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.542872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9da8-1095-46cd-a977-6eab1afab551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.565321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-521d9da8-1095-46cd-a977-6eab1afab551 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:43.566656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-521d9da8-1095-46cd-a977-6eab1afab551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:43.642367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1115/2241] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:43.651017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431e4a-df15-4e4b-96c4-8ebf818b89f2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.651304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431e4a-df15-4e4b-96c4-8ebf818b89f2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.651697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431e4a-df15-4e4b-96c4-8ebf818b89f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.688175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-37431e4a-df15-4e4b-96c4-8ebf818b89f2 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:43.689441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-37431e4a-df15-4e4b-96c4-8ebf818b89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:43.723463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1127/2242] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:43.728254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d9da8-1095-46cd-a977-6eab1afab551 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:43.740432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d9da8-1095-46cd-a977-6eab1afab551 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:43.745975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799561f1-2440-453a-9f5d-66ead19b66aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:43.747083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799561f1-2440-453a-9f5d-66ead19b66aa admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.747651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799561f1-2440-453a-9f5d-66ead19b66aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.748316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799561f1-2440-453a-9f5d-66ead19b66aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.750314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799561f1-2440-453a-9f5d-66ead19b66aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:43.750890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799561f1-2440-453a-9f5d-66ead19b66aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:43.752853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799561f1-2440-453a-9f5d-66ead19b66aa admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:43.768796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1116/2243] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:43.773179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37431e4a-df15-4e4b-96c4-8ebf818b89f2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:43.784956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37431e4a-df15-4e4b-96c4-8ebf818b89f2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:43.789941 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59383f6-2bbe-478c-9610-655b7389f66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:43.791047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59383f6-2bbe-478c-9610-655b7389f66a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.791474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59383f6-2bbe-478c-9610-655b7389f66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.791746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59383f6-2bbe-478c-9610-655b7389f66a admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.792581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59383f6-2bbe-478c-9610-655b7389f66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:43.795950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59383f6-2bbe-478c-9610-655b7389f66a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:43.831836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1128/2244] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:43 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:43.835474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799561f1-2440-453a-9f5d-66ead19b66aa admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:43.852002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799561f1-2440-453a-9f5d-66ead19b66aa admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:43.899033 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:43.977363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf5b40-6652-40cc-836e-3c96b2c7ffb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:43.977975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf5b40-6652-40cc-836e-3c96b2c7ffb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.978254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf5b40-6652-40cc-836e-3c96b2c7ffb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.978797 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf5b40-6652-40cc-836e-3c96b2c7ffb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.979174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf5b40-6652-40cc-836e-3c96b2c7ffb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:43.981222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf5b40-6652-40cc-836e-3c96b2c7ffb1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.005491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1117/2245] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:43 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:44.010478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59383f6-2bbe-478c-9610-655b7389f66a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.022374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59383f6-2bbe-478c-9610-655b7389f66a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.027132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e34dc-849a-4eba-9b4a-c0395dcc76f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.027903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e34dc-849a-4eba-9b4a-c0395dcc76f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.028189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e34dc-849a-4eba-9b4a-c0395dcc76f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.028461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e34dc-849a-4eba-9b4a-c0395dcc76f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.029296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e34dc-849a-4eba-9b4a-c0395dcc76f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.030908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e34dc-849a-4eba-9b4a-c0395dcc76f4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.046334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1129/2246] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:44.047273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf5b40-6652-40cc-836e-3c96b2c7ffb1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.058337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf5b40-6652-40cc-836e-3c96b2c7ffb1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.065038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285f645-7946-41de-9e65-72fdb4e30d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.065760 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285f645-7946-41de-9e65-72fdb4e30d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.066140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285f645-7946-41de-9e65-72fdb4e30d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.066508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285f645-7946-41de-9e65-72fdb4e30d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.067264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285f645-7946-41de-9e65-72fdb4e30d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.068932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285f645-7946-41de-9e65-72fdb4e30d3f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.081339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1118/2247] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:44.085919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e34dc-849a-4eba-9b4a-c0395dcc76f4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.097687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e34dc-849a-4eba-9b4a-c0395dcc76f4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.107911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19843f34-c6c1-4ca0-982b-3f2fbf4ab0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.108229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19843f34-c6c1-4ca0-982b-3f2fbf4ab0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.108514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19843f34-c6c1-4ca0-982b-3f2fbf4ab0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.108830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19843f34-c6c1-4ca0-982b-3f2fbf4ab0ee admin admin] PATH_INFO: `/v3/projects/573b07907a53441d9ce0b965ecd12780/users/e0e03e2a856c4de08c38d556b3d844df/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.126738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19843f34-c6c1-4ca0-982b-3f2fbf4ab0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=573b07907a53441d9ce0b965ecd12780, user_id=e0e03e2a856c4de08c38d556b3d844df, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.129554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19843f34-c6c1-4ca0-982b-3f2fbf4ab0ee admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.156361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72bd8b-609c-4dbb-9edf-dc192712e07a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.156752 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72bd8b-609c-4dbb-9edf-dc192712e07a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.157104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72bd8b-609c-4dbb-9edf-dc192712e07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.159906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1130/2248] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:44 2022] PUT /identity/v3/projects/573b07907a53441d9ce0b965ecd12780/users/e0e03e2a856c4de08c38d556b3d844df/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:44.196114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cf72bd8b-609c-4dbb-9edf-dc192712e07a None None] MFA Rules not processed for user `e0e03e2a856c4de08c38d556b3d844df`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:44.197376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cf72bd8b-609c-4dbb-9edf-dc192712e07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:44.277139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1119/2249] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:44.288836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19843f34-c6c1-4ca0-982b-3f2fbf4ab0ee admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.290583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-19843f34-c6c1-4ca0-982b-3f2fbf4ab0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:44.352290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19843f34-c6c1-4ca0-982b-3f2fbf4ab0ee admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.356044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed529e95-0a0d-4a04-8b18-86a3575e8761 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a47e77cc1942618a3584862042c5f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.356590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed529e95-0a0d-4a04-8b18-86a3575e8761 service glance] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.356954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed529e95-0a0d-4a04-8b18-86a3575e8761 service glance] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.357306 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed529e95-0a0d-4a04-8b18-86a3575e8761 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.365522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed529e95-0a0d-4a04-8b18-86a3575e8761 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.366964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed529e95-0a0d-4a04-8b18-86a3575e8761 service glance] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.375811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1131/2250] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:48.277914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68874f97-5bd2-49a4-a20d-538f76c48ffa None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.278360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68874f97-5bd2-49a4-a20d-538f76c48ffa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.278493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68874f97-5bd2-49a4-a20d-538f76c48ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.297920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-68874f97-5bd2-49a4-a20d-538f76c48ffa None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:48.299115 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-68874f97-5bd2-49a4-a20d-538f76c48ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:48.358961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1120/2251] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:02:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:48.402312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed529e95-0a0d-4a04-8b18-86a3575e8761 service glance] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.403745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ed529e95-0a0d-4a04-8b18-86a3575e8761 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:48.468901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:48.497298 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed529e95-0a0d-4a04-8b18-86a3575e8761 service glance] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.503076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6eda0b-bb14-4db4-8ba6-279ac14c5a11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.503684 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6eda0b-bb14-4db4-8ba6-279ac14c5a11 service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.503961 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6eda0b-bb14-4db4-8ba6-279ac14c5a11 service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.504229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6eda0b-bb14-4db4-8ba6-279ac14c5a11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.512787 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6eda0b-bb14-4db4-8ba6-279ac14c5a11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.515261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6eda0b-bb14-4db4-8ba6-279ac14c5a11 service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.523972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1132/2252] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:02:49.947159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68874f97-5bd2-49a4-a20d-538f76c48ffa None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.948572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-68874f97-5bd2-49a4-a20d-538f76c48ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:49.983730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:50.016652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68874f97-5bd2-49a4-a20d-538f76c48ffa None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.020493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaae110-2ac0-4a68-afed-61c507d492e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.020918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaae110-2ac0-4a68-afed-61c507d492e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.021236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaae110-2ac0-4a68-afed-61c507d492e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.021506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaae110-2ac0-4a68-afed-61c507d492e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.023479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bfaae110-2ac0-4a68-afed-61c507d492e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:50.109764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaae110-2ac0-4a68-afed-61c507d492e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.111251 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaae110-2ac0-4a68-afed-61c507d492e5 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.123207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1121/2253] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:55.241430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6eda0b-bb14-4db4-8ba6-279ac14c5a11 service swift] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:55.246326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3f6eda0b-bb14-4db4-8ba6-279ac14c5a11 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:55.333864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6eda0b-bb14-4db4-8ba6-279ac14c5a11 service swift] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:55.343693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:55.344434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.344933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.345241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] PATH_INFO: `/v3/users/e84a45debfc246c28ccf6d4be5129232` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.346418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84a45debfc246c28ccf6d4be5129232)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:55.348371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:55.433755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84a45debfc246c28ccf6d4be5129232'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:55.450112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84a45debfc246c28ccf6d4be5129232'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:55.461672 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84a45debfc246c28ccf6d4be5129232'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:55.462733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84a45debfc246c28ccf6d4be5129232'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:55.521654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84a45debfc246c28ccf6d4be5129232'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:55.540134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1133/2254] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:55 2022] DELETE /identity/v3/users/e84a45debfc246c28ccf6d4be5129232 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:55.546225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d8169-2437-4eb9-8b88-49baf3c50de8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.546767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d8169-2437-4eb9-8b88-49baf3c50de8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.546767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d8169-2437-4eb9-8b88-49baf3c50de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.570987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f29d8169-2437-4eb9-8b88-49baf3c50de8 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:55.572286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f29d8169-2437-4eb9-8b88-49baf3c50de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:55.650964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1122/2255] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:55.662089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:55.662907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:55.728318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96420a54-59b0-4c9f-b2b3-19598b155246 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:55.732771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6220dd7-3e60-4e65-abec-8b17a5d072df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:55.733531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6220dd7-3e60-4e65-abec-8b17a5d072df service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.733809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6220dd7-3e60-4e65-abec-8b17a5d072df service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.734078 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6220dd7-3e60-4e65-abec-8b17a5d072df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.744406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6220dd7-3e60-4e65-abec-8b17a5d072df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:55.746150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6220dd7-3e60-4e65-abec-8b17a5d072df service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:55.754423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:55.770661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1134/2256] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:56.105436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d8169-2437-4eb9-8b88-49baf3c50de8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:56.107779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f29d8169-2437-4eb9-8b88-49baf3c50de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:56.129612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d8169-2437-4eb9-8b88-49baf3c50de8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:56.141593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:56.143099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:56.143637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:56.144426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] PATH_INFO: `/v3/projects/1971fa3358044f5ba43c88a3abdabb5f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:56.146222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1971fa3358044f5ba43c88a3abdabb5f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:56.148652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:56.219115 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1971fa3358044f5ba43c88a3abdabb5f', 'request_id': 'req-96a21a68-2e40-4e70-99d5-8cec23a07cc7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.227279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1971fa3358044f5ba43c88a3abdabb5f', 'request_id': 'req-96a21a68-2e40-4e70-99d5-8cec23a07cc7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.237388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1971fa3358044f5ba43c88a3abdabb5f', 'request_id': 'req-96a21a68-2e40-4e70-99d5-8cec23a07cc7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.250490 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] The token cache is being invalidate because project 1971fa3358044f5ba43c88a3abdabb5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:56.250910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.251653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.253708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1123/2257] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:56 2022] DELETE /identity/v3/projects/1971fa3358044f5ba43c88a3abdabb5f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:56.257503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6220dd7-3e60-4e65-abec-8b17a5d072df service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:56.271422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6220dd7-3e60-4e65-abec-8b17a5d072df service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:56.342117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:56.342760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:56.343072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:56.343346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] PATH_INFO: `/v3/users/48723ba440504645877fb41a4ee89400` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:56.344494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48723ba440504645877fb41a4ee89400)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:56.346512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:56.397437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48723ba440504645877fb41a4ee89400'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.406494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48723ba440504645877fb41a4ee89400'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.414361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48723ba440504645877fb41a4ee89400'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.415039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48723ba440504645877fb41a4ee89400'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.431087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-13059235-468a-430e-abcb-f80fdf8bb635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48723ba440504645877fb41a4ee89400'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.437030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1135/2258] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:56 2022] DELETE /identity/v3/users/48723ba440504645877fb41a4ee89400 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:56.791165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:56.795303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:56.805315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:56.833296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a21a68-2e40-4e70-99d5-8cec23a07cc7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:56.906133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:56.906887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:56.907178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:56.907468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] PATH_INFO: `/v3/projects/c0458d0f9bf2475fb03fcd1d71120ff5` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:56.908654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0458d0f9bf2475fb03fcd1d71120ff5)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:56.910619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:56.965202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0458d0f9bf2475fb03fcd1d71120ff5', 'request_id': 'req-678d0c63-a0ba-4f34-918f-40ce1238f614'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.972876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0458d0f9bf2475fb03fcd1d71120ff5', 'request_id': 'req-678d0c63-a0ba-4f34-918f-40ce1238f614'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.982597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0458d0f9bf2475fb03fcd1d71120ff5', 'request_id': 'req-678d0c63-a0ba-4f34-918f-40ce1238f614'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.993922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] The token cache is being invalidate because project c0458d0f9bf2475fb03fcd1d71120ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:56.994203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.994844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-678d0c63-a0ba-4f34-918f-40ce1238f614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:56.996550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1124/2259] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:56 2022] DELETE /identity/v3/projects/c0458d0f9bf2475fb03fcd1d71120ff5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:57.003221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4194a1-2292-4a81-8c2f-ec433b6ec10a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.003587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4194a1-2292-4a81-8c2f-ec433b6ec10a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.004015 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4194a1-2292-4a81-8c2f-ec433b6ec10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.026339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9e4194a1-2292-4a81-8c2f-ec433b6ec10a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:57.027570 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9e4194a1-2292-4a81-8c2f-ec433b6ec10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:57.099114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1136/2260] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:57.107963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4510-c01d-4e87-92dc-77e3723a38ed None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.108286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4510-c01d-4e87-92dc-77e3723a38ed None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.108532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4510-c01d-4e87-92dc-77e3723a38ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.131191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-bbaf4510-c01d-4e87-92dc-77e3723a38ed None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:57.132410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bbaf4510-c01d-4e87-92dc-77e3723a38ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:57.141442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1125/2261] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:57.145901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4194a1-2292-4a81-8c2f-ec433b6ec10a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.157999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4194a1-2292-4a81-8c2f-ec433b6ec10a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.162958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4d6b6-1c63-43af-8a17-6d559df64102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.163646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4d6b6-1c63-43af-8a17-6d559df64102 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.163925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4d6b6-1c63-43af-8a17-6d559df64102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.164192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4d6b6-1c63-43af-8a17-6d559df64102 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.164859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4d6b6-1c63-43af-8a17-6d559df64102 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:57.165317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4d6b6-1c63-43af-8a17-6d559df64102 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.167333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4d6b6-1c63-43af-8a17-6d559df64102 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.184361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1137/2262] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:57.189111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf4510-c01d-4e87-92dc-77e3723a38ed None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.199630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf4510-c01d-4e87-92dc-77e3723a38ed None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.207736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4159f-aaec-4d3a-823b-e27d90f82550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.208563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4159f-aaec-4d3a-823b-e27d90f82550 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.208820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4159f-aaec-4d3a-823b-e27d90f82550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.209179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4159f-aaec-4d3a-823b-e27d90f82550 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.210935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4159f-aaec-4d3a-823b-e27d90f82550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.213554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4159f-aaec-4d3a-823b-e27d90f82550 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.272595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1126/2263] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:57 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:57.276967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4d6b6-1c63-43af-8a17-6d559df64102 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.284518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:57.289053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4d6b6-1c63-43af-8a17-6d559df64102 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.383140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c799d83-fd7a-4413-b6d9-4d5852ef9782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.384242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c799d83-fd7a-4413-b6d9-4d5852ef9782 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.384750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c799d83-fd7a-4413-b6d9-4d5852ef9782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.385275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c799d83-fd7a-4413-b6d9-4d5852ef9782 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.385958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c799d83-fd7a-4413-b6d9-4d5852ef9782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.388560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c799d83-fd7a-4413-b6d9-4d5852ef9782 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.417413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1138/2264] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:57 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:57.422066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4159f-aaec-4d3a-823b-e27d90f82550 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.437434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4159f-aaec-4d3a-823b-e27d90f82550 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.442838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee015e3-077a-4f58-9e60-81b25ed95a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.443453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee015e3-077a-4f58-9e60-81b25ed95a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.443734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee015e3-077a-4f58-9e60-81b25ed95a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.444008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee015e3-077a-4f58-9e60-81b25ed95a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.444659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee015e3-077a-4f58-9e60-81b25ed95a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.446458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee015e3-077a-4f58-9e60-81b25ed95a53 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.461950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1127/2265] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:57.466798 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c799d83-fd7a-4413-b6d9-4d5852ef9782 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.480901 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c799d83-fd7a-4413-b6d9-4d5852ef9782 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.502649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d21b2-c974-4cf1-9027-83319af7535d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.503279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d21b2-c974-4cf1-9027-83319af7535d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.503557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d21b2-c974-4cf1-9027-83319af7535d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.503836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d21b2-c974-4cf1-9027-83319af7535d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.504537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d21b2-c974-4cf1-9027-83319af7535d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.506276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d21b2-c974-4cf1-9027-83319af7535d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.525201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1139/2266] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:57.529857 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee015e3-077a-4f58-9e60-81b25ed95a53 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.541568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee015e3-077a-4f58-9e60-81b25ed95a53 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.548012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10637aac-1860-426c-9e20-6c400a6dec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.548805 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10637aac-1860-426c-9e20-6c400a6dec78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.549690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10637aac-1860-426c-9e20-6c400a6dec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.550078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10637aac-1860-426c-9e20-6c400a6dec78 admin admin] PATH_INFO: `/v3/projects/833c90525bc845d0b88393a6d003d2d7/users/f0f9e08f5e114aee9852a5302f40d7ce/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.568149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10637aac-1860-426c-9e20-6c400a6dec78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=833c90525bc845d0b88393a6d003d2d7, user_id=f0f9e08f5e114aee9852a5302f40d7ce, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.570839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10637aac-1860-426c-9e20-6c400a6dec78 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.585823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1128/2267] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:57 2022] PUT /identity/v3/projects/833c90525bc845d0b88393a6d003d2d7/users/f0f9e08f5e114aee9852a5302f40d7ce/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:57.600728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e475a-f004-4836-a9ad-e3cee497b6a6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.601304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e475a-f004-4836-a9ad-e3cee497b6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.601976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e475a-f004-4836-a9ad-e3cee497b6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.649536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1b5e475a-f004-4836-a9ad-e3cee497b6a6 None None] MFA Rules not processed for user `f0f9e08f5e114aee9852a5302f40d7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:57.651581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1b5e475a-f004-4836-a9ad-e3cee497b6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:57.775717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1140/2268] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:02:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:57.783631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10637aac-1860-426c-9e20-6c400a6dec78 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.785986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-10637aac-1860-426c-9e20-6c400a6dec78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:57.833468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:57.870274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10637aac-1860-426c-9e20-6c400a6dec78 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.876599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9d45f-91d1-4f86-b373-caf39fdb27d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.877447 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d45f-91d1-4f86-b373-caf39fdb27d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.877936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d45f-91d1-4f86-b373-caf39fdb27d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.878311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d45f-91d1-4f86-b373-caf39fdb27d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.890421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9d45f-91d1-4f86-b373-caf39fdb27d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.892074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9d45f-91d1-4f86-b373-caf39fdb27d7 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.928243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1129/2269] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:00.776615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e475a-f004-4836-a9ad-e3cee497b6a6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.854488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e475a-f004-4836-a9ad-e3cee497b6a6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.860618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.861411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.861815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.862188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] PATH_INFO: `/v3/users/f0f9e08f5e114aee9852a5302f40d7ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.863448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f9e08f5e114aee9852a5302f40d7ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.865640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.895348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:00.956923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f9e08f5e114aee9852a5302f40d7ce'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.969830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f9e08f5e114aee9852a5302f40d7ce'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.979114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f9e08f5e114aee9852a5302f40d7ce'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.979905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f9e08f5e114aee9852a5302f40d7ce'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.995622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f9e08f5e114aee9852a5302f40d7ce'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:01.009296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1141/2270] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:00 2022] DELETE /identity/v3/users/f0f9e08f5e114aee9852a5302f40d7ce => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:01.012375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaead6e-fc08-4453-af31-8cb94d942c3c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.012948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaead6e-fc08-4453-af31-8cb94d942c3c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.013110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaead6e-fc08-4453-af31-8cb94d942c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.033545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0aaead6e-fc08-4453-af31-8cb94d942c3c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:01.035509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0aaead6e-fc08-4453-af31-8cb94d942c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:01.112683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1130/2271] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:01.123583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.126187 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:01.189477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa910-707d-41ca-97c0-eebb4fa3b8f0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.193171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd2536-860a-441b-8636-f0c7cb916efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.193789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd2536-860a-441b-8636-f0c7cb916efb service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.194098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd2536-860a-441b-8636-f0c7cb916efb service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.194332 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd2536-860a-441b-8636-f0c7cb916efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.206838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd2536-860a-441b-8636-f0c7cb916efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.208202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd2536-860a-441b-8636-f0c7cb916efb service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.219820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1142/2272] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:01.524446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaead6e-fc08-4453-af31-8cb94d942c3c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.526163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0aaead6e-fc08-4453-af31-8cb94d942c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:01.539449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaead6e-fc08-4453-af31-8cb94d942c3c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.556426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.556835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.557328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.557783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] PATH_INFO: `/v3/projects/833c90525bc845d0b88393a6d003d2d7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.559333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=833c90525bc845d0b88393a6d003d2d7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.561910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.618774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '833c90525bc845d0b88393a6d003d2d7', 'request_id': 'req-d61c3f57-9362-4618-8e73-d1fb9aa251ea'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:01.626000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '833c90525bc845d0b88393a6d003d2d7', 'request_id': 'req-d61c3f57-9362-4618-8e73-d1fb9aa251ea'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:01.637085 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '833c90525bc845d0b88393a6d003d2d7', 'request_id': 'req-d61c3f57-9362-4618-8e73-d1fb9aa251ea'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:01.650647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] The token cache is being invalidate because project 833c90525bc845d0b88393a6d003d2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:01.650957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:01.651583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d61c3f57-9362-4618-8e73-d1fb9aa251ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:01.653215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1131/2273] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:01 2022] DELETE /identity/v3/projects/833c90525bc845d0b88393a6d003d2d7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:01.659313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c7b30-834a-4ae8-ae51-9bf49a517414 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.659586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c7b30-834a-4ae8-ae51-9bf49a517414 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.659866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c7b30-834a-4ae8-ae51-9bf49a517414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.681088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-458c7b30-834a-4ae8-ae51-9bf49a517414 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:01.682302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-458c7b30-834a-4ae8-ae51-9bf49a517414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:01.753741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1143/2274] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:01.761459 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1aee-0ca8-4e06-be84-ae2ab3158d91 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.761755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1aee-0ca8-4e06-be84-ae2ab3158d91 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.762025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1aee-0ca8-4e06-be84-ae2ab3158d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.782784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2aaa1aee-0ca8-4e06-be84-ae2ab3158d91 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:01.784346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2aaa1aee-0ca8-4e06-be84-ae2ab3158d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:01.792536 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1132/2275] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:01.797804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c7b30-834a-4ae8-ae51-9bf49a517414 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.807982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c7b30-834a-4ae8-ae51-9bf49a517414 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.813481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb6d6-03ab-4e27-a317-b81d02ab4d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.814082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb6d6-03ab-4e27-a317-b81d02ab4d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.814358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb6d6-03ab-4e27-a317-b81d02ab4d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.814628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb6d6-03ab-4e27-a317-b81d02ab4d58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.815277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abeb6d6-03ab-4e27-a317-b81d02ab4d58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:01.815582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abeb6d6-03ab-4e27-a317-b81d02ab4d58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.817361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abeb6d6-03ab-4e27-a317-b81d02ab4d58 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.832218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1144/2276] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:01.836377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa1aee-0ca8-4e06-be84-ae2ab3158d91 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.847424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa1aee-0ca8-4e06-be84-ae2ab3158d91 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.852289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5e9aa-56ab-4c57-968f-b90d91947d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.852892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5e9aa-56ab-4c57-968f-b90d91947d25 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.853187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5e9aa-56ab-4c57-968f-b90d91947d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.853464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5e9aa-56ab-4c57-968f-b90d91947d25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.854092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5e9aa-56ab-4c57-968f-b90d91947d25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.856010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5e9aa-56ab-4c57-968f-b90d91947d25 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.912542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1133/2277] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:01 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:01.913441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb6d6-03ab-4e27-a317-b81d02ab4d58 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.924432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb6d6-03ab-4e27-a317-b81d02ab4d58 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.968103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:02.024047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc582c6-c8b5-4e0b-9a8f-e58432b2c0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:02.024876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc582c6-c8b5-4e0b-9a8f-e58432b2c0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:02.025378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc582c6-c8b5-4e0b-9a8f-e58432b2c0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:02.025726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc582c6-c8b5-4e0b-9a8f-e58432b2c0f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:02.026685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc582c6-c8b5-4e0b-9a8f-e58432b2c0f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:02.028972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc582c6-c8b5-4e0b-9a8f-e58432b2c0f3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:02.062988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1145/2278] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:01 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:02.068173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5e9aa-56ab-4c57-968f-b90d91947d25 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:02.084438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5e9aa-56ab-4c57-968f-b90d91947d25 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:02.091740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18751e0a-e780-430b-9ae6-35e249b95f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:02.093379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18751e0a-e780-430b-9ae6-35e249b95f3f admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:02.093629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18751e0a-e780-430b-9ae6-35e249b95f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:02.094273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18751e0a-e780-430b-9ae6-35e249b95f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:02.095554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18751e0a-e780-430b-9ae6-35e249b95f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:02.098199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18751e0a-e780-430b-9ae6-35e249b95f3f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:02.115488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1134/2279] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:02.119225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc582c6-c8b5-4e0b-9a8f-e58432b2c0f3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:02.135010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc582c6-c8b5-4e0b-9a8f-e58432b2c0f3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:02.145597 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de5fd2-db6b-4630-b9f3-9dc4e8c8336f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:02.146478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de5fd2-db6b-4630-b9f3-9dc4e8c8336f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:02.146923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de5fd2-db6b-4630-b9f3-9dc4e8c8336f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:02.147198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de5fd2-db6b-4630-b9f3-9dc4e8c8336f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:02.147867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de5fd2-db6b-4630-b9f3-9dc4e8c8336f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:02.149854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de5fd2-db6b-4630-b9f3-9dc4e8c8336f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:02.166264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1146/2280] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:02.167275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18751e0a-e780-430b-9ae6-35e249b95f3f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:02.180593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18751e0a-e780-430b-9ae6-35e249b95f3f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:02.200791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5fe01-2e49-4ee0-ad43-27279ee6bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:02.201666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5fe01-2e49-4ee0-ad43-27279ee6bce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:02.201814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5fe01-2e49-4ee0-ad43-27279ee6bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:02.202042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5fe01-2e49-4ee0-ad43-27279ee6bce7 admin admin] PATH_INFO: `/v3/projects/edf1fbe8a6824bdfb1dbb03893c7f46f/users/f25d18022a674617b7974ed74b51ddf8/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:02.220801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:02.224437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5fe01-2e49-4ee0-ad43-27279ee6bce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf1fbe8a6824bdfb1dbb03893c7f46f, user_id=f25d18022a674617b7974ed74b51ddf8, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:02.236030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5fe01-2e49-4ee0-ad43-27279ee6bce7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:02.273414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1135/2281] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:02 2022] PUT /identity/v3/projects/edf1fbe8a6824bdfb1dbb03893c7f46f/users/f25d18022a674617b7974ed74b51ddf8/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:02.284259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0ab10-ff4e-4c85-a22b-f90d6a2a879c None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:02.284691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0ab10-ff4e-4c85-a22b-f90d6a2a879c None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:02.285105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0ab10-ff4e-4c85-a22b-f90d6a2a879c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:02.324046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9db0ab10-ff4e-4c85-a22b-f90d6a2a879c None None] MFA Rules not processed for user `f25d18022a674617b7974ed74b51ddf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:02.325395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9db0ab10-ff4e-4c85-a22b-f90d6a2a879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:02.447422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1147/2282] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:02.460200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5fe01-2e49-4ee0-ad43-27279ee6bce7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:02.462167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-69b5fe01-2e49-4ee0-ad43-27279ee6bce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:02.537019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5fe01-2e49-4ee0-ad43-27279ee6bce7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:02.540838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce185210-2d34-4227-a607-f52992a614f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:02.541532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce185210-2d34-4227-a607-f52992a614f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:02.541806 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce185210-2d34-4227-a607-f52992a614f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:02.542077 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce185210-2d34-4227-a607-f52992a614f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:02.552289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce185210-2d34-4227-a607-f52992a614f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:02.553714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce185210-2d34-4227-a607-f52992a614f0 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:02.563566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1136/2283] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:06.264192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff1be-fadd-4721-b440-88a1f683a3ff None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:06.264663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff1be-fadd-4721-b440-88a1f683a3ff None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:06.264782 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff1be-fadd-4721-b440-88a1f683a3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:06.284204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-98dff1be-fadd-4721-b440-88a1f683a3ff None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:06.285466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-98dff1be-fadd-4721-b440-88a1f683a3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:06.351720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1148/2284] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:03:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:06.368299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce185210-2d34-4227-a607-f52992a614f0 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:06.370066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ce185210-2d34-4227-a607-f52992a614f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:06.456672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce185210-2d34-4227-a607-f52992a614f0 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:06.466633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a38f9-4f17-4b18-ba76-0341078b340f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:06.467478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a38f9-4f17-4b18-ba76-0341078b340f service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:06.467820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a38f9-4f17-4b18-ba76-0341078b340f service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:06.468189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a38f9-4f17-4b18-ba76-0341078b340f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:06.477224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a38f9-4f17-4b18-ba76-0341078b340f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:06.478858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a38f9-4f17-4b18-ba76-0341078b340f service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:06.489197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1137/2285] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:03:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:03:07.182276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01749eb9-3f68-4e18-bdb3-ea7d2bf0059f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:07.182818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01749eb9-3f68-4e18-bdb3-ea7d2bf0059f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:07.183276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01749eb9-3f68-4e18-bdb3-ea7d2bf0059f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:07.205517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-01749eb9-3f68-4e18-bdb3-ea7d2bf0059f None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:07.207086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-01749eb9-3f68-4e18-bdb3-ea7d2bf0059f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:07.218051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1149/2286] 10.222.0.78 () {58 vars in 967 bytes} [Tue Aug 30 01:03:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:07.251032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a38f9-4f17-4b18-ba76-0341078b340f service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:07.278441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a38f9-4f17-4b18-ba76-0341078b340f service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:07.294999 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d761a-82b7-40c3-9612-c20380d20987 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:07.295721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d761a-82b7-40c3-9612-c20380d20987 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:07.296335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d761a-82b7-40c3-9612-c20380d20987 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:07.296458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d761a-82b7-40c3-9612-c20380d20987 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:07.312913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d761a-82b7-40c3-9612-c20380d20987 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:07.314820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d761a-82b7-40c3-9612-c20380d20987 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:07.329874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1138/2287] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:03:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:03:07.691251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cedec-87f8-40b4-a8c9-3841b49f2bc0 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:07.692172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cedec-87f8-40b4-a8c9-3841b49f2bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:07.692554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cedec-87f8-40b4-a8c9-3841b49f2bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:07.725970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ce6cedec-87f8-40b4-a8c9-3841b49f2bc0 None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:07.727768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ce6cedec-87f8-40b4-a8c9-3841b49f2bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:07.741490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1150/2288] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:03:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:07.758069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d761a-82b7-40c3-9612-c20380d20987 service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:07.771947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d761a-82b7-40c3-9612-c20380d20987 service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:07.780187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8e2c0-7e9b-4d76-9320-6fc7886cb2a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:07.781145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8e2c0-7e9b-4d76-9320-6fc7886cb2a6 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:07.781695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8e2c0-7e9b-4d76-9320-6fc7886cb2a6 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:07.781858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8e2c0-7e9b-4d76-9320-6fc7886cb2a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:07.794996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8e2c0-7e9b-4d76-9320-6fc7886cb2a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:07.796856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8e2c0-7e9b-4d76-9320-6fc7886cb2a6 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:07.806619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1139/2289] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:03:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:03:08.241146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828d8a-1631-4f01-a3e5-4f79a10dd186 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:08.243153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828d8a-1631-4f01-a3e5-4f79a10dd186 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:08.243592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828d8a-1631-4f01-a3e5-4f79a10dd186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:08.275796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4e828d8a-1631-4f01-a3e5-4f79a10dd186 None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:08.277216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4e828d8a-1631-4f01-a3e5-4f79a10dd186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:08.288009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1151/2290] 10.222.0.78 () {58 vars in 967 bytes} [Tue Aug 30 01:03:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:08.318335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8e2c0-7e9b-4d76-9320-6fc7886cb2a6 service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:08.332755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8e2c0-7e9b-4d76-9320-6fc7886cb2a6 service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:08.339596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd294ece-8074-4fd6-98f0-e1398666a0ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:08.340591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd294ece-8074-4fd6-98f0-e1398666a0ff service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:08.341455 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd294ece-8074-4fd6-98f0-e1398666a0ff service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:08.341837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd294ece-8074-4fd6-98f0-e1398666a0ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:08.351579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd294ece-8074-4fd6-98f0-e1398666a0ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:08.353199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd294ece-8074-4fd6-98f0-e1398666a0ff service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:08.363874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1140/2291] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:03:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:03:11.125840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e828d8a-1631-4f01-a3e5-4f79a10dd186 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.127262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4e828d8a-1631-4f01-a3e5-4f79a10dd186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:11.146282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.195525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e828d8a-1631-4f01-a3e5-4f79a10dd186 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.200663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.201449 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.201737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.202008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] PATH_INFO: `/v3/users/a2c60f3426484bc0baa3107ae3568724` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.203331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c60f3426484bc0baa3107ae3568724)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.205300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.270869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c60f3426484bc0baa3107ae3568724'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.281035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c60f3426484bc0baa3107ae3568724'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.288451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c60f3426484bc0baa3107ae3568724'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.289344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c60f3426484bc0baa3107ae3568724'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.304054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c60f3426484bc0baa3107ae3568724'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.312112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1152/2292] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:11 2022] DELETE /identity/v3/users/a2c60f3426484bc0baa3107ae3568724 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:11.316612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54180-29c2-4d71-8522-2ec7be395d8a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.317168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54180-29c2-4d71-8522-2ec7be395d8a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.317168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54180-29c2-4d71-8522-2ec7be395d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.336593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1ab54180-29c2-4d71-8522-2ec7be395d8a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:11.337848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1ab54180-29c2-4d71-8522-2ec7be395d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:11.380581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.400128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1141/2293] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:11.410539 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.411955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:11.469576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290ee0b-5879-4143-ad50-b02de4e2f7c3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.473054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881ed4f-b49d-4dd3-9e6d-90bdf816b1fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.473702 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881ed4f-b49d-4dd3-9e6d-90bdf816b1fa service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.473983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881ed4f-b49d-4dd3-9e6d-90bdf816b1fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.474256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881ed4f-b49d-4dd3-9e6d-90bdf816b1fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.483622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881ed4f-b49d-4dd3-9e6d-90bdf816b1fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.485042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881ed4f-b49d-4dd3-9e6d-90bdf816b1fa service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.494435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1153/2294] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:11.666146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54180-29c2-4d71-8522-2ec7be395d8a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.667585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1ab54180-29c2-4d71-8522-2ec7be395d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:11.680392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54180-29c2-4d71-8522-2ec7be395d8a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.686535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.687174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.687423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.687743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] PATH_INFO: `/v3/projects/8a8b8077287845c1815a82a8fdbf51b6` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.688887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a8b8077287845c1815a82a8fdbf51b6)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.691817 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.743701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a8b8077287845c1815a82a8fdbf51b6', 'request_id': 'req-55689f1a-a515-4592-be8e-3715ed376896'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.751485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a8b8077287845c1815a82a8fdbf51b6', 'request_id': 'req-55689f1a-a515-4592-be8e-3715ed376896'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.759703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a8b8077287845c1815a82a8fdbf51b6', 'request_id': 'req-55689f1a-a515-4592-be8e-3715ed376896'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.772400 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] The token cache is being invalidate because project 8a8b8077287845c1815a82a8fdbf51b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:11.772769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.773466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-55689f1a-a515-4592-be8e-3715ed376896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:11.774962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1142/2295] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:11 2022] DELETE /identity/v3/projects/8a8b8077287845c1815a82a8fdbf51b6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:11.780041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53b563-13cd-497e-8e4d-c551741a3440 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.780328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53b563-13cd-497e-8e4d-c551741a3440 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.780622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53b563-13cd-497e-8e4d-c551741a3440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.800514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4c53b563-13cd-497e-8e4d-c551741a3440 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:11.801780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4c53b563-13cd-497e-8e4d-c551741a3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:11.833960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.864804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1154/2296] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:11.873444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b56ebd-669f-4209-a98e-62d4409b3981 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.873764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b56ebd-669f-4209-a98e-62d4409b3981 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.874011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b56ebd-669f-4209-a98e-62d4409b3981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.894879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-22b56ebd-669f-4209-a98e-62d4409b3981 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:11.896159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-22b56ebd-669f-4209-a98e-62d4409b3981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:11.904690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1143/2297] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:11.908809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53b563-13cd-497e-8e4d-c551741a3440 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.919495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53b563-13cd-497e-8e4d-c551741a3440 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.923979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafa8b1-6525-440e-8489-39a98e891537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.924564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafa8b1-6525-440e-8489-39a98e891537 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.924835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafa8b1-6525-440e-8489-39a98e891537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.925127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafa8b1-6525-440e-8489-39a98e891537 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.925767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafa8b1-6525-440e-8489-39a98e891537 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:11.926063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafa8b1-6525-440e-8489-39a98e891537 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.927658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafa8b1-6525-440e-8489-39a98e891537 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.938983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1155/2298] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:11.943183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b56ebd-669f-4209-a98e-62d4409b3981 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.952577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b56ebd-669f-4209-a98e-62d4409b3981 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.958300 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964d356-79f6-4603-b5b9-2b883a516526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.958938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964d356-79f6-4603-b5b9-2b883a516526 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.959221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964d356-79f6-4603-b5b9-2b883a516526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.959495 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964d356-79f6-4603-b5b9-2b883a516526 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.960158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964d356-79f6-4603-b5b9-2b883a516526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.962098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964d356-79f6-4603-b5b9-2b883a516526 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.990900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1144/2299] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:11 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:11.994956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafa8b1-6525-440e-8489-39a98e891537 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.004897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafa8b1-6525-440e-8489-39a98e891537 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.062297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cb023-c5dc-4fa3-922c-a587f172129c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.062950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cb023-c5dc-4fa3-922c-a587f172129c admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.063235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cb023-c5dc-4fa3-922c-a587f172129c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.063510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cb023-c5dc-4fa3-922c-a587f172129c admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.064178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cb023-c5dc-4fa3-922c-a587f172129c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.066214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cb023-c5dc-4fa3-922c-a587f172129c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.088057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1156/2300] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:11 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:12.092488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964d356-79f6-4603-b5b9-2b883a516526 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.102813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964d356-79f6-4603-b5b9-2b883a516526 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.107631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fdc4c-9831-4bb6-9bd4-2c43c0c3ddf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.108219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fdc4c-9831-4bb6-9bd4-2c43c0c3ddf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.108498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fdc4c-9831-4bb6-9bd4-2c43c0c3ddf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.108766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fdc4c-9831-4bb6-9bd4-2c43c0c3ddf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.109442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fdc4c-9831-4bb6-9bd4-2c43c0c3ddf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.111069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fdc4c-9831-4bb6-9bd4-2c43c0c3ddf3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.121083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1145/2301] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:12.125443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cb023-c5dc-4fa3-922c-a587f172129c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.135225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cb023-c5dc-4fa3-922c-a587f172129c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.140598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b4d66-4a4d-45bf-8666-893499913ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.141216 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b4d66-4a4d-45bf-8666-893499913ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.141492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b4d66-4a4d-45bf-8666-893499913ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.141758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b4d66-4a4d-45bf-8666-893499913ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.142394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b4d66-4a4d-45bf-8666-893499913ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.143958 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b4d66-4a4d-45bf-8666-893499913ef0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.154365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1157/2302] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:12.158647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fdc4c-9831-4bb6-9bd4-2c43c0c3ddf3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.168731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fdc4c-9831-4bb6-9bd4-2c43c0c3ddf3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.173352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb69e6-23f5-4975-9016-5f7bf07627d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.173979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb69e6-23f5-4975-9016-5f7bf07627d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.174253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb69e6-23f5-4975-9016-5f7bf07627d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.174523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb69e6-23f5-4975-9016-5f7bf07627d8 admin admin] PATH_INFO: `/v3/projects/fca43fd867154911badb413c7046befd/users/0aa5b198f4e14d52a369248f1e6c0f47/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.189232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb69e6-23f5-4975-9016-5f7bf07627d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca43fd867154911badb413c7046befd, user_id=0aa5b198f4e14d52a369248f1e6c0f47, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.191857 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb69e6-23f5-4975-9016-5f7bf07627d8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.205004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1146/2303] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:12 2022] PUT /identity/v3/projects/fca43fd867154911badb413c7046befd/users/0aa5b198f4e14d52a369248f1e6c0f47/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:12.210322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3ab09-771d-447a-bce8-bf1919b2e30a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.210627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3ab09-771d-447a-bce8-bf1919b2e30a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.210898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3ab09-771d-447a-bce8-bf1919b2e30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.229957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-21a3ab09-771d-447a-bce8-bf1919b2e30a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:12.231695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-21a3ab09-771d-447a-bce8-bf1919b2e30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:12.316181 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1158/2304] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:12.326451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb69e6-23f5-4975-9016-5f7bf07627d8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.328302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3dcb69e6-23f5-4975-9016-5f7bf07627d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:12.384845 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb69e6-23f5-4975-9016-5f7bf07627d8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.388690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb34ae-dc23-4d52-bfc4-d2454713f4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.389314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb34ae-dc23-4d52-bfc4-d2454713f4ca service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.389597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb34ae-dc23-4d52-bfc4-d2454713f4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.389871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb34ae-dc23-4d52-bfc4-d2454713f4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.398827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb34ae-dc23-4d52-bfc4-d2454713f4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.400232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb34ae-dc23-4d52-bfc4-d2454713f4ca service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.409773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1147/2305] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:03:19.840385 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b803b5a-8903-40dd-8038-dd0f91fea415 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.841022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b803b5a-8903-40dd-8038-dd0f91fea415 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.841407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b803b5a-8903-40dd-8038-dd0f91fea415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.875383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9b803b5a-8903-40dd-8038-dd0f91fea415 None None] MFA Rules not processed for user `0aa5b198f4e14d52a369248f1e6c0f47`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:19.876733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9b803b5a-8903-40dd-8038-dd0f91fea415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:19.913281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:19.948357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1159/2306] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:19.952888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb34ae-dc23-4d52-bfc4-d2454713f4ca service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.964237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb34ae-dc23-4d52-bfc4-d2454713f4ca service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.968213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a2eee-5cb9-4b4e-9339-4502c0315e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.968816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a2eee-5cb9-4b4e-9339-4502c0315e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.969113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a2eee-5cb9-4b4e-9339-4502c0315e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.969383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a2eee-5cb9-4b4e-9339-4502c0315e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.970031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a2eee-5cb9-4b4e-9339-4502c0315e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:19.971905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a2eee-5cb9-4b4e-9339-4502c0315e48 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:19.976615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.002730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1148/2307] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:19 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:20.007010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b803b5a-8903-40dd-8038-dd0f91fea415 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.020018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b803b5a-8903-40dd-8038-dd0f91fea415 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.094053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482892e-8c83-4fb2-95c1-342128ba942a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.094745 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482892e-8c83-4fb2-95c1-342128ba942a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.095093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482892e-8c83-4fb2-95c1-342128ba942a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.095430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482892e-8c83-4fb2-95c1-342128ba942a admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.096125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482892e-8c83-4fb2-95c1-342128ba942a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.098278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482892e-8c83-4fb2-95c1-342128ba942a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.128332 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1160/2308] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:20 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:20.129375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a2eee-5cb9-4b4e-9339-4502c0315e48 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.142984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a2eee-5cb9-4b4e-9339-4502c0315e48 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.149949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c0ce4-abbc-4d1a-a498-c0b9f4b6bcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.150790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c0ce4-abbc-4d1a-a498-c0b9f4b6bcb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.151220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c0ce4-abbc-4d1a-a498-c0b9f4b6bcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.151640 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c0ce4-abbc-4d1a-a498-c0b9f4b6bcb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.152435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c0ce4-abbc-4d1a-a498-c0b9f4b6bcb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.154855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c0ce4-abbc-4d1a-a498-c0b9f4b6bcb4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.171966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1149/2309] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:20.175772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482892e-8c83-4fb2-95c1-342128ba942a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.188797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482892e-8c83-4fb2-95c1-342128ba942a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.194712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c3ddf-ce8d-4ddb-9c99-75595e1be009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.195975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c3ddf-ce8d-4ddb-9c99-75595e1be009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.196458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c3ddf-ce8d-4ddb-9c99-75595e1be009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.196940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c3ddf-ce8d-4ddb-9c99-75595e1be009 admin admin] PATH_INFO: `/v3/projects/f7ddbc07b2f149d2867b67c837c60afd/users/472cc26b48ad4976bcd28395fbeefa1e/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.216555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c3ddf-ce8d-4ddb-9c99-75595e1be009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ddbc07b2f149d2867b67c837c60afd, user_id=472cc26b48ad4976bcd28395fbeefa1e, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.220114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c3ddf-ce8d-4ddb-9c99-75595e1be009 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.266462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1161/2310] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:20 2022] PUT /identity/v3/projects/f7ddbc07b2f149d2867b67c837c60afd/users/472cc26b48ad4976bcd28395fbeefa1e/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:20.271083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c0ce4-abbc-4d1a-a498-c0b9f4b6bcb4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.286260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c0ce4-abbc-4d1a-a498-c0b9f4b6bcb4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.390407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a2b19-c97f-4a6a-9e29-d336d342689e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.391086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a2b19-c97f-4a6a-9e29-d336d342689e admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.391506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a2b19-c97f-4a6a-9e29-d336d342689e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.391795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a2b19-c97f-4a6a-9e29-d336d342689e admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.392576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a2b19-c97f-4a6a-9e29-d336d342689e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.394353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a2b19-c97f-4a6a-9e29-d336d342689e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.413978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1150/2311] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:20.418574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c3ddf-ce8d-4ddb-9c99-75595e1be009 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.429269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c3ddf-ce8d-4ddb-9c99-75595e1be009 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.434458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128df236-59a0-45f8-8f17-42755d734786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.435173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128df236-59a0-45f8-8f17-42755d734786 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.435374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128df236-59a0-45f8-8f17-42755d734786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.435643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128df236-59a0-45f8-8f17-42755d734786 admin admin] PATH_INFO: `/v3/projects/f7ddbc07b2f149d2867b67c837c60afd/users/472cc26b48ad4976bcd28395fbeefa1e/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.437585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128df236-59a0-45f8-8f17-42755d734786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ddbc07b2f149d2867b67c837c60afd, user_id=472cc26b48ad4976bcd28395fbeefa1e, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.440252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128df236-59a0-45f8-8f17-42755d734786 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.451521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1162/2312] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:20 2022] PUT /identity/v3/projects/f7ddbc07b2f149d2867b67c837c60afd/users/472cc26b48ad4976bcd28395fbeefa1e/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:20.796011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de135bdc-a8d4-4ae1-95eb-45b398673246 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.796305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de135bdc-a8d4-4ae1-95eb-45b398673246 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.796876 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de135bdc-a8d4-4ae1-95eb-45b398673246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.844387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-de135bdc-a8d4-4ae1-95eb-45b398673246 None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:20.846717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-de135bdc-a8d4-4ae1-95eb-45b398673246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:21.005710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1151/2313] 10.222.0.78 () {58 vars in 967 bytes} [Tue Aug 30 01:03:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:21.013725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128df236-59a0-45f8-8f17-42755d734786 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.016353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-128df236-59a0-45f8-8f17-42755d734786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:21.124879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128df236-59a0-45f8-8f17-42755d734786 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.133966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bae272-9ff2-4575-8e50-c62be867124e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.134873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bae272-9ff2-4575-8e50-c62be867124e service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.135316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bae272-9ff2-4575-8e50-c62be867124e service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.135733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bae272-9ff2-4575-8e50-c62be867124e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.148533 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bae272-9ff2-4575-8e50-c62be867124e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.150709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bae272-9ff2-4575-8e50-c62be867124e service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.164108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1163/2314] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:03:24.848436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de135bdc-a8d4-4ae1-95eb-45b398673246 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:24.849927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-de135bdc-a8d4-4ae1-95eb-45b398673246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:24.974725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de135bdc-a8d4-4ae1-95eb-45b398673246 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:24.982226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:24.983593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.983840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.984159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] PATH_INFO: `/v3/users/fed877b0a569455597dfbb8b98ae0bc0` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.985918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed877b0a569455597dfbb8b98ae0bc0)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:24.988734 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.039165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.091417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed877b0a569455597dfbb8b98ae0bc0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.113099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed877b0a569455597dfbb8b98ae0bc0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.113835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed877b0a569455597dfbb8b98ae0bc0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.114468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed877b0a569455597dfbb8b98ae0bc0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.152319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d67cf2e9-0a89-495e-83fd-02489cd6758e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed877b0a569455597dfbb8b98ae0bc0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.174576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1152/2315] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:24 2022] DELETE /identity/v3/users/fed877b0a569455597dfbb8b98ae0bc0 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:25.605237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bae272-9ff2-4575-8e50-c62be867124e service swift] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:25.618804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bae272-9ff2-4575-8e50-c62be867124e service swift] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:25.655510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.721295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:25.722748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.723034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.723470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] PATH_INFO: `/v3/projects/2c18e1da199c437b809a45aada0b0e0e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.724810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c18e1da199c437b809a45aada0b0e0e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:25.727315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.814436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c18e1da199c437b809a45aada0b0e0e', 'request_id': 'req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.832092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c18e1da199c437b809a45aada0b0e0e', 'request_id': 'req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.843719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c18e1da199c437b809a45aada0b0e0e', 'request_id': 'req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.868036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] The token cache is being invalidate because project 2c18e1da199c437b809a45aada0b0e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:25.868607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.869480 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c53c1bd2-20b5-46f7-8a46-61a1c46a6cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:25.873447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1164/2316] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:25 2022] DELETE /identity/v3/projects/2c18e1da199c437b809a45aada0b0e0e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:25.878156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffcaf9-b1fc-40d0-97f0-f8a65eb752d1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.878839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffcaf9-b1fc-40d0-97f0-f8a65eb752d1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.878839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffcaf9-b1fc-40d0-97f0-f8a65eb752d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.909139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b3ffcaf9-b1fc-40d0-97f0-f8a65eb752d1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:25.910460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b3ffcaf9-b1fc-40d0-97f0-f8a65eb752d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:26.003110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1153/2317] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:26.011869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a68b63-c4dc-4e3c-be72-6d04644d4f65 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.012404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a68b63-c4dc-4e3c-be72-6d04644d4f65 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.012703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a68b63-c4dc-4e3c-be72-6d04644d4f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.061233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-21a68b63-c4dc-4e3c-be72-6d04644d4f65 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:26.062944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-21a68b63-c4dc-4e3c-be72-6d04644d4f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:26.083915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1165/2318] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:26.085035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffcaf9-b1fc-40d0-97f0-f8a65eb752d1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.102796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffcaf9-b1fc-40d0-97f0-f8a65eb752d1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.109834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80f307-239e-422f-bd1f-c551867c72d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.110908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80f307-239e-422f-bd1f-c551867c72d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.111402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80f307-239e-422f-bd1f-c551867c72d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.111889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80f307-239e-422f-bd1f-c551867c72d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.112747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80f307-239e-422f-bd1f-c551867c72d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:26.113285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80f307-239e-422f-bd1f-c551867c72d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.115061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80f307-239e-422f-bd1f-c551867c72d5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.133474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1154/2319] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:26.137777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a68b63-c4dc-4e3c-be72-6d04644d4f65 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.148560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a68b63-c4dc-4e3c-be72-6d04644d4f65 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.155648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd2a67-480c-476e-a27c-c8e9457a9cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.156275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd2a67-480c-476e-a27c-c8e9457a9cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.156556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd2a67-480c-476e-a27c-c8e9457a9cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.156832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd2a67-480c-476e-a27c-c8e9457a9cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.157528 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd2a67-480c-476e-a27c-c8e9457a9cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.159463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd2a67-480c-476e-a27c-c8e9457a9cc0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.206250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1166/2320] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:26 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:26.210095 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80f307-239e-422f-bd1f-c551867c72d5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.227087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80f307-239e-422f-bd1f-c551867c72d5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.264369 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cf02e-5790-4aee-a27a-f0ff2a64d889 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.264670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cf02e-5790-4aee-a27a-f0ff2a64d889 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.264957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cf02e-5790-4aee-a27a-f0ff2a64d889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.332456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-641cf02e-5790-4aee-a27a-f0ff2a64d889 None None] MFA Rules not processed for user `472cc26b48ad4976bcd28395fbeefa1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:26.334712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-641cf02e-5790-4aee-a27a-f0ff2a64d889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:26.372566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f8a96-2ddc-494b-b8ec-84cab1d88196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.373303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f8a96-2ddc-494b-b8ec-84cab1d88196 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.373660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f8a96-2ddc-494b-b8ec-84cab1d88196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.374030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f8a96-2ddc-494b-b8ec-84cab1d88196 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.374755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f8a96-2ddc-494b-b8ec-84cab1d88196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.376816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f8a96-2ddc-494b-b8ec-84cab1d88196 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.410664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1155/2321] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:26 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:26.414277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f8a96-2ddc-494b-b8ec-84cab1d88196 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.427750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f8a96-2ddc-494b-b8ec-84cab1d88196 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.432948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a146d08-8254-4f2e-a82d-cec5e3b415c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.433930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a146d08-8254-4f2e-a82d-cec5e3b415c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.434291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a146d08-8254-4f2e-a82d-cec5e3b415c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.434652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a146d08-8254-4f2e-a82d-cec5e3b415c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.435365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a146d08-8254-4f2e-a82d-cec5e3b415c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.437044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a146d08-8254-4f2e-a82d-cec5e3b415c5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.444153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:26.454508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1156/2322] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:03:26.455862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a146d08-8254-4f2e-a82d-cec5e3b415c5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.460192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1167/2323] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:26.468843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a146d08-8254-4f2e-a82d-cec5e3b415c5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.473634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cf02e-5790-4aee-a27a-f0ff2a64d889 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.474638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b3d2-0b29-47f3-805f-587f2a463d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.475148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-641cf02e-5790-4aee-a27a-f0ff2a64d889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:26.475314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b3d2-0b29-47f3-805f-587f2a463d27 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.475591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b3d2-0b29-47f3-805f-587f2a463d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.475853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b3d2-0b29-47f3-805f-587f2a463d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.476491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b3d2-0b29-47f3-805f-587f2a463d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.478119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b3d2-0b29-47f3-805f-587f2a463d27 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.482327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:26.494612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1157/2324] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:26.500198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b3d2-0b29-47f3-805f-587f2a463d27 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.512343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b3d2-0b29-47f3-805f-587f2a463d27 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.517678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196cb5c-a8b0-424e-a38d-941693dcfe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.518410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196cb5c-a8b0-424e-a38d-941693dcfe82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.518699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196cb5c-a8b0-424e-a38d-941693dcfe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.518966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196cb5c-a8b0-424e-a38d-941693dcfe82 admin admin] PATH_INFO: `/v3/projects/155fcb4c06cf40cebfa8443dffe4c4eb/users/bcd7186ed9c745c8b8ff68a72f1f50d1/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.542140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196cb5c-a8b0-424e-a38d-941693dcfe82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155fcb4c06cf40cebfa8443dffe4c4eb, user_id=bcd7186ed9c745c8b8ff68a72f1f50d1, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.545932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196cb5c-a8b0-424e-a38d-941693dcfe82 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.552778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cf02e-5790-4aee-a27a-f0ff2a64d889 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.557869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3b14d-0084-4777-8794-791c19399031 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a47e77cc1942618a3584862042c5f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.559459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b14d-0084-4777-8794-791c19399031 service glance] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.560405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b14d-0084-4777-8794-791c19399031 service glance] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.560688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b14d-0084-4777-8794-791c19399031 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.563128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f8e3b14d-0084-4777-8794-791c19399031 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:26.565387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1158/2325] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:26 2022] PUT /identity/v3/projects/155fcb4c06cf40cebfa8443dffe4c4eb/users/bcd7186ed9c745c8b8ff68a72f1f50d1/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:26.575503 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6065-a1fe-42fd-88ee-0df891082d12 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.575804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6065-a1fe-42fd-88ee-0df891082d12 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.576079 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6065-a1fe-42fd-88ee-0df891082d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.611718 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c8bb6065-a1fe-42fd-88ee-0df891082d12 None None] MFA Rules not processed for user `bcd7186ed9c745c8b8ff68a72f1f50d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:26.612958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c8bb6065-a1fe-42fd-88ee-0df891082d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:26.688628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3b14d-0084-4777-8794-791c19399031 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.692444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3b14d-0084-4777-8794-791c19399031 service glance] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.710966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1159/2326] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:26.711375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1168/2327] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:03:26.720613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6065-a1fe-42fd-88ee-0df891082d12 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.722538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c8bb6065-a1fe-42fd-88ee-0df891082d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:26.842821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6065-a1fe-42fd-88ee-0df891082d12 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.847872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc2801-0ada-4de2-bcfa-cdeaf76a19a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.849236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc2801-0ada-4de2-bcfa-cdeaf76a19a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.849548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc2801-0ada-4de2-bcfa-cdeaf76a19a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.850209 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc2801-0ada-4de2-bcfa-cdeaf76a19a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.865879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc2801-0ada-4de2-bcfa-cdeaf76a19a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.867508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc2801-0ada-4de2-bcfa-cdeaf76a19a6 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.879086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1160/2328] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:26.890006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c3852-92ea-41a4-8fdd-63454b301e1f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.890006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c3852-92ea-41a4-8fdd-63454b301e1f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.890006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c3852-92ea-41a4-8fdd-63454b301e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.926945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-310c3852-92ea-41a4-8fdd-63454b301e1f None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:26.928186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-310c3852-92ea-41a4-8fdd-63454b301e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:27.007917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1169/2329] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:03:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:27.023290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc2801-0ada-4de2-bcfa-cdeaf76a19a6 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.027732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-80fc2801-0ada-4de2-bcfa-cdeaf76a19a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:27.110998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:27.129042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc2801-0ada-4de2-bcfa-cdeaf76a19a6 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.135645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194e409-8870-42a0-af63-2a22af6b6fbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.136204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194e409-8870-42a0-af63-2a22af6b6fbd service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.136520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194e409-8870-42a0-af63-2a22af6b6fbd service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.136985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194e409-8870-42a0-af63-2a22af6b6fbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.148030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194e409-8870-42a0-af63-2a22af6b6fbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.149677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194e409-8870-42a0-af63-2a22af6b6fbd service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.160737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1161/2330] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:03:29.041149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c3852-92ea-41a4-8fdd-63454b301e1f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.043388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-310c3852-92ea-41a4-8fdd-63454b301e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:29.141068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c3852-92ea-41a4-8fdd-63454b301e1f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.148065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.148718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.148996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.149356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] PATH_INFO: `/v3/users/e0e03e2a856c4de08c38d556b3d844df` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.150544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e03e2a856c4de08c38d556b3d844df)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.154851 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.157749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.226098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e03e2a856c4de08c38d556b3d844df'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.246604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e03e2a856c4de08c38d556b3d844df'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.267411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e03e2a856c4de08c38d556b3d844df'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.268473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e03e2a856c4de08c38d556b3d844df'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.346007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e03e2a856c4de08c38d556b3d844df'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.358492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1170/2331] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:29 2022] DELETE /identity/v3/users/e0e03e2a856c4de08c38d556b3d844df => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:29.360403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960611b-aab0-4d71-aac7-c58776d68d80 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.360913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960611b-aab0-4d71-aac7-c58776d68d80 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.361607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960611b-aab0-4d71-aac7-c58776d68d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.419319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3960611b-aab0-4d71-aac7-c58776d68d80 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:29.421159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3960611b-aab0-4d71-aac7-c58776d68d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:29.520721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1162/2332] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:29.536568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.538148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:29.637940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0012c-685e-4962-8860-a245680934fe admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.650708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b7ff4-a42a-4ece-a21c-fad32434944d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.651435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b7ff4-a42a-4ece-a21c-fad32434944d service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.651783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b7ff4-a42a-4ece-a21c-fad32434944d service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.652156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b7ff4-a42a-4ece-a21c-fad32434944d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.662835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b7ff4-a42a-4ece-a21c-fad32434944d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.664520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b7ff4-a42a-4ece-a21c-fad32434944d service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.676666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1171/2333] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:30.250638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960611b-aab0-4d71-aac7-c58776d68d80 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.255159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3960611b-aab0-4d71-aac7-c58776d68d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:30.307214 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960611b-aab0-4d71-aac7-c58776d68d80 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.372050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.376071 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.376361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.380439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] PATH_INFO: `/v3/projects/573b07907a53441d9ce0b965ecd12780` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.391632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] RBAC: Authorizing `identity:delete_project(project_id=573b07907a53441d9ce0b965ecd12780)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.393181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.604034 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573b07907a53441d9ce0b965ecd12780', 'request_id': 'req-f09d644a-a3f8-485c-98da-44983005e06f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:30.627515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573b07907a53441d9ce0b965ecd12780', 'request_id': 'req-f09d644a-a3f8-485c-98da-44983005e06f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:30.653833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573b07907a53441d9ce0b965ecd12780', 'request_id': 'req-f09d644a-a3f8-485c-98da-44983005e06f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:30.672258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] The token cache is being invalidate because project 573b07907a53441d9ce0b965ecd12780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:30.673533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:30.676579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:30.682182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1163/2334] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:30 2022] DELETE /identity/v3/projects/573b07907a53441d9ce0b965ecd12780 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:32.310108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0329f5-3ee5-40eb-a94f-9a25836953fd None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.310642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0329f5-3ee5-40eb-a94f-9a25836953fd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.310642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0329f5-3ee5-40eb-a94f-9a25836953fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.359125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7e0329f5-3ee5-40eb-a94f-9a25836953fd None None] MFA Rules not processed for user `cc7cf306aadb45d1b65252b53c86b612`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:32.361706 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7e0329f5-3ee5-40eb-a94f-9a25836953fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:32.496949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1172/2335] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:39.224745 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.226935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:39.335481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d644a-a3f8-485c-98da-44983005e06f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.340670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.341347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.341633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.342043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] PATH_INFO: `/v3/users/bcd7186ed9c745c8b8ff68a72f1f50d1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.343508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd7186ed9c745c8b8ff68a72f1f50d1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.345646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.362489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:39.387531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8922e55-9c43-467a-9f56-bc59868a9fa2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.392167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8922e55-9c43-467a-9f56-bc59868a9fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.392433 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8922e55-9c43-467a-9f56-bc59868a9fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.415648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f8922e55-9c43-467a-9f56-bc59868a9fa2 None None] MFA Rules not processed for user `0aa5b198f4e14d52a369248f1e6c0f47`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:39.417865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f8922e55-9c43-467a-9f56-bc59868a9fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:39.418691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd7186ed9c745c8b8ff68a72f1f50d1'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.426365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd7186ed9c745c8b8ff68a72f1f50d1'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.427389 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd7186ed9c745c8b8ff68a72f1f50d1'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.428708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd7186ed9c745c8b8ff68a72f1f50d1'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.442977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-73662acb-5d9c-4f98-af91-4f89de718b97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd7186ed9c745c8b8ff68a72f1f50d1'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.443144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:39.457401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1164/2336] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:39 2022] DELETE /identity/v3/users/bcd7186ed9c745c8b8ff68a72f1f50d1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:39.461952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94cbc3-d53a-4891-bb04-fbe922eb3c1c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.462235 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94cbc3-d53a-4891-bb04-fbe922eb3c1c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.462502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94cbc3-d53a-4891-bb04-fbe922eb3c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.478273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1173/2337] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:39.483147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-aa94cbc3-d53a-4891-bb04-fbe922eb3c1c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:39.484379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-aa94cbc3-d53a-4891-bb04-fbe922eb3c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:39.546093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1165/2338] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:39.555885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8922e55-9c43-467a-9f56-bc59868a9fa2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.557399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f8922e55-9c43-467a-9f56-bc59868a9fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:39.617765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8922e55-9c43-467a-9f56-bc59868a9fa2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.668720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1b3a-ba10-4fcd-8b67-69f3bdd40be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.669279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1b3a-ba10-4fcd-8b67-69f3bdd40be5 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.669556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1b3a-ba10-4fcd-8b67-69f3bdd40be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.669825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1b3a-ba10-4fcd-8b67-69f3bdd40be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.679458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a1b3a-ba10-4fcd-8b67-69f3bdd40be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.681117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a1b3a-ba10-4fcd-8b67-69f3bdd40be5 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.693564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1174/2339] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:40.012351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94cbc3-d53a-4891-bb04-fbe922eb3c1c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.013983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-aa94cbc3-d53a-4891-bb04-fbe922eb3c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:40.035213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94cbc3-d53a-4891-bb04-fbe922eb3c1c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.044164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.044940 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.045379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.045540 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] PATH_INFO: `/v3/projects/155fcb4c06cf40cebfa8443dffe4c4eb` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.046924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=155fcb4c06cf40cebfa8443dffe4c4eb)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.059578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.161120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155fcb4c06cf40cebfa8443dffe4c4eb', 'request_id': 'req-ca0455f6-b5d6-414a-a738-8995db700c75'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:40.169012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155fcb4c06cf40cebfa8443dffe4c4eb', 'request_id': 'req-ca0455f6-b5d6-414a-a738-8995db700c75'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:40.185631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155fcb4c06cf40cebfa8443dffe4c4eb', 'request_id': 'req-ca0455f6-b5d6-414a-a738-8995db700c75'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:40.211774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] The token cache is being invalidate because project 155fcb4c06cf40cebfa8443dffe4c4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:40.216624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:40.217835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:40.220590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1166/2340] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:40 2022] DELETE /identity/v3/projects/155fcb4c06cf40cebfa8443dffe4c4eb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:41.760126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdac616-5435-4413-a65d-7f9209912f35 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.762098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdac616-5435-4413-a65d-7f9209912f35 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.762372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdac616-5435-4413-a65d-7f9209912f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.789497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-cfdac616-5435-4413-a65d-7f9209912f35 None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:41.790862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cfdac616-5435-4413-a65d-7f9209912f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:41.854548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1175/2341] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:03:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:41.866963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.868937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:41.884172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.947937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0455f6-b5d6-414a-a738-8995db700c75 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.953965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d96fe-29a4-4979-98a4-18196cd93d11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.954405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d96fe-29a4-4979-98a4-18196cd93d11 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.954684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d96fe-29a4-4979-98a4-18196cd93d11 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.954956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d96fe-29a4-4979-98a4-18196cd93d11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.963452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d96fe-29a4-4979-98a4-18196cd93d11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.965220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d96fe-29a4-4979-98a4-18196cd93d11 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.974314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1167/2342] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:03:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:03:45.389984 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdac616-5435-4413-a65d-7f9209912f35 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.391422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cfdac616-5435-4413-a65d-7f9209912f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:45.452011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:45.509179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdac616-5435-4413-a65d-7f9209912f35 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:45.518661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:45.519559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.519993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.521169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] PATH_INFO: `/v3/users/f25d18022a674617b7974ed74b51ddf8` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.522633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25d18022a674617b7974ed74b51ddf8)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:45.525897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:45.621401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25d18022a674617b7974ed74b51ddf8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:45.630755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25d18022a674617b7974ed74b51ddf8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:45.648301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25d18022a674617b7974ed74b51ddf8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:45.649111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25d18022a674617b7974ed74b51ddf8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:45.669051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25d18022a674617b7974ed74b51ddf8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:45.675878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1176/2343] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:45 2022] DELETE /identity/v3/users/f25d18022a674617b7974ed74b51ddf8 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:45.680478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7b1da-86cb-460b-9b54-4420cb12e750 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.680974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7b1da-86cb-460b-9b54-4420cb12e750 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.681067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7b1da-86cb-460b-9b54-4420cb12e750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.701260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b3d7b1da-86cb-460b-9b54-4420cb12e750 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:45.702716 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b3d7b1da-86cb-460b-9b54-4420cb12e750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:45.774328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1168/2344] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:45.782205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.783840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:45.848851 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eb88c-17d9-4c9a-aad8-2d3d80b1dc40 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:45.855894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ec370-4d4b-42b6-a815-0cdd6ecc92e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:45.856563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ec370-4d4b-42b6-a815-0cdd6ecc92e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.856788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ec370-4d4b-42b6-a815-0cdd6ecc92e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.857063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ec370-4d4b-42b6-a815-0cdd6ecc92e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.866731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ec370-4d4b-42b6-a815-0cdd6ecc92e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:45.868171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ec370-4d4b-42b6-a815-0cdd6ecc92e9 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:45.881018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1177/2345] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:46.192914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7b1da-86cb-460b-9b54-4420cb12e750 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.194899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b3d7b1da-86cb-460b-9b54-4420cb12e750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:46.216615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7b1da-86cb-460b-9b54-4420cb12e750 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.230544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.231222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.231498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.231769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] PATH_INFO: `/v3/projects/edf1fbe8a6824bdfb1dbb03893c7f46f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.232973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf1fbe8a6824bdfb1dbb03893c7f46f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.236169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.345281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf1fbe8a6824bdfb1dbb03893c7f46f', 'request_id': 'req-c7e23051-6828-406d-8bd7-b6c4f6f88d88'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:46.364342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf1fbe8a6824bdfb1dbb03893c7f46f', 'request_id': 'req-c7e23051-6828-406d-8bd7-b6c4f6f88d88'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:46.393939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf1fbe8a6824bdfb1dbb03893c7f46f', 'request_id': 'req-c7e23051-6828-406d-8bd7-b6c4f6f88d88'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:46.409566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] The token cache is being invalidate because project edf1fbe8a6824bdfb1dbb03893c7f46f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:46.410596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:46.411912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c7e23051-6828-406d-8bd7-b6c4f6f88d88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:46.413493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1169/2346] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:46 2022] DELETE /identity/v3/projects/edf1fbe8a6824bdfb1dbb03893c7f46f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:46.419747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4154e2-002c-40f4-b82b-0d97009ed533 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.420651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4154e2-002c-40f4-b82b-0d97009ed533 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.421190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4154e2-002c-40f4-b82b-0d97009ed533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.444138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ef4154e2-002c-40f4-b82b-0d97009ed533 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:46.445785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ef4154e2-002c-40f4-b82b-0d97009ed533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:46.537546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:46.550551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1178/2347] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:46.557900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831fb86-e102-4d5c-9368-07c9679fc9ad None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.558208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831fb86-e102-4d5c-9368-07c9679fc9ad None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.558479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831fb86-e102-4d5c-9368-07c9679fc9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.586904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a831fb86-e102-4d5c-9368-07c9679fc9ad None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:46.588134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a831fb86-e102-4d5c-9368-07c9679fc9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:46.597636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1170/2348] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:46.605477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4154e2-002c-40f4-b82b-0d97009ed533 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.629022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4154e2-002c-40f4-b82b-0d97009ed533 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.643098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78b695-b17a-4e54-a450-24e6bbff94de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.645852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78b695-b17a-4e54-a450-24e6bbff94de admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.646455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78b695-b17a-4e54-a450-24e6bbff94de admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.646914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78b695-b17a-4e54-a450-24e6bbff94de admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.650472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78b695-b17a-4e54-a450-24e6bbff94de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:46.651780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78b695-b17a-4e54-a450-24e6bbff94de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.656638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78b695-b17a-4e54-a450-24e6bbff94de admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.680442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1179/2349] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:46.685408 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831fb86-e102-4d5c-9368-07c9679fc9ad None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.695380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831fb86-e102-4d5c-9368-07c9679fc9ad None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.709058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8116e4-6090-4b2d-a292-b1a89bb9766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.709957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8116e4-6090-4b2d-a292-b1a89bb9766e admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.710394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8116e4-6090-4b2d-a292-b1a89bb9766e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.710832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8116e4-6090-4b2d-a292-b1a89bb9766e admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.711496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8116e4-6090-4b2d-a292-b1a89bb9766e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.713822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8116e4-6090-4b2d-a292-b1a89bb9766e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.741303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:46.752172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1171/2350] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:46 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:46.758642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78b695-b17a-4e54-a450-24e6bbff94de admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.776771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78b695-b17a-4e54-a450-24e6bbff94de admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.851873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d98b0-073f-4c72-b9a4-b6e157a8d0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.852626 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d98b0-073f-4c72-b9a4-b6e157a8d0d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.853592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d98b0-073f-4c72-b9a4-b6e157a8d0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.853592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d98b0-073f-4c72-b9a4-b6e157a8d0d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.853885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d98b0-073f-4c72-b9a4-b6e157a8d0d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.856663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d98b0-073f-4c72-b9a4-b6e157a8d0d1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.883110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1180/2351] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:46 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:46.887648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8116e4-6090-4b2d-a292-b1a89bb9766e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.900748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8116e4-6090-4b2d-a292-b1a89bb9766e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.925728 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd08dc-98b1-4a05-942a-bce7c8395b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.926663 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd08dc-98b1-4a05-942a-bce7c8395b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.926943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd08dc-98b1-4a05-942a-bce7c8395b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.927214 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd08dc-98b1-4a05-942a-bce7c8395b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.928507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd08dc-98b1-4a05-942a-bce7c8395b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.930828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd08dc-98b1-4a05-942a-bce7c8395b77 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.941959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1172/2352] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:46.945729 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d98b0-073f-4c72-b9a4-b6e157a8d0d1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.957658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d98b0-073f-4c72-b9a4-b6e157a8d0d1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.962741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce682c-a6a7-4464-9a26-cd89c73279a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.963357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce682c-a6a7-4464-9a26-cd89c73279a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.964215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce682c-a6a7-4464-9a26-cd89c73279a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.964963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce682c-a6a7-4464-9a26-cd89c73279a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.965205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce682c-a6a7-4464-9a26-cd89c73279a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.966818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce682c-a6a7-4464-9a26-cd89c73279a3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.984495 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1181/2353] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:03:46.988563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd08dc-98b1-4a05-942a-bce7c8395b77 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.999142 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd08dc-98b1-4a05-942a-bce7c8395b77 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:47.007796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ad744-f7f3-4a2b-98e4-7fc9fb757bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:47.007796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ad744-f7f3-4a2b-98e4-7fc9fb757bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.007796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ad744-f7f3-4a2b-98e4-7fc9fb757bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.007796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ad744-f7f3-4a2b-98e4-7fc9fb757bb5 admin admin] PATH_INFO: `/v3/projects/a6380baa172744fa9d7ba5b81f847906/users/d467895d586a43719d2abdf4105187f8/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.025042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ad744-f7f3-4a2b-98e4-7fc9fb757bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6380baa172744fa9d7ba5b81f847906, user_id=d467895d586a43719d2abdf4105187f8, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:47.028581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ad744-f7f3-4a2b-98e4-7fc9fb757bb5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:47.044446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1173/2354] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:46 2022] PUT /identity/v3/projects/a6380baa172744fa9d7ba5b81f847906/users/d467895d586a43719d2abdf4105187f8/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:47.050184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ed083-e71f-4f9a-802c-1fef6588d172 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.050823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ed083-e71f-4f9a-802c-1fef6588d172 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.051128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ed083-e71f-4f9a-802c-1fef6588d172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.071749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c14ed083-e71f-4f9a-802c-1fef6588d172 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:47.074417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c14ed083-e71f-4f9a-802c-1fef6588d172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:47.139275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1182/2355] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:47.148986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ad744-f7f3-4a2b-98e4-7fc9fb757bb5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:47.150769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-222ad744-f7f3-4a2b-98e4-7fc9fb757bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:47.217130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ad744-f7f3-4a2b-98e4-7fc9fb757bb5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:47.222873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237198da-c09e-47fb-98ae-1837f0433021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:47.223942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237198da-c09e-47fb-98ae-1837f0433021 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.224453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237198da-c09e-47fb-98ae-1837f0433021 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.224902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237198da-c09e-47fb-98ae-1837f0433021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.236767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237198da-c09e-47fb-98ae-1837f0433021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:47.238232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237198da-c09e-47fb-98ae-1837f0433021 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:47.251526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1174/2356] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:53.374142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c3d60-f518-4f2d-a95b-67399b5f85c1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.374683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c3d60-f518-4f2d-a95b-67399b5f85c1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.374683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c3d60-f518-4f2d-a95b-67399b5f85c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.409631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a65c3d60-f518-4f2d-a95b-67399b5f85c1 None None] MFA Rules not processed for user `d467895d586a43719d2abdf4105187f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:53.411009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a65c3d60-f518-4f2d-a95b-67399b5f85c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.474769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.493160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1183/2357] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:03:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:53.495834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237198da-c09e-47fb-98ae-1837f0433021 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.508210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237198da-c09e-47fb-98ae-1837f0433021 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.513137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a97a2b-ff4d-4fd6-9a5a-55d8678cf407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.513799 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a97a2b-ff4d-4fd6-9a5a-55d8678cf407 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.514087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a97a2b-ff4d-4fd6-9a5a-55d8678cf407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.514553 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a97a2b-ff4d-4fd6-9a5a-55d8678cf407 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.515002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a97a2b-ff4d-4fd6-9a5a-55d8678cf407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.516894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a97a2b-ff4d-4fd6-9a5a-55d8678cf407 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.550823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1175/2358] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:53 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:53.554816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c3d60-f518-4f2d-a95b-67399b5f85c1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.569817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c3d60-f518-4f2d-a95b-67399b5f85c1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.655905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a963c-7615-4c3c-a446-9fb01e47d543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.656916 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a963c-7615-4c3c-a446-9fb01e47d543 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.657340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a963c-7615-4c3c-a446-9fb01e47d543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.657609 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a963c-7615-4c3c-a446-9fb01e47d543 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.658301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a963c-7615-4c3c-a446-9fb01e47d543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.660448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a963c-7615-4c3c-a446-9fb01e47d543 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.695322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1184/2359] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:53 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:53.696691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a97a2b-ff4d-4fd6-9a5a-55d8678cf407 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.714128 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a97a2b-ff4d-4fd6-9a5a-55d8678cf407 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.729875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f0488-51b2-45d8-b7ad-b995517c5422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.730791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f0488-51b2-45d8-b7ad-b995517c5422 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.731075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f0488-51b2-45d8-b7ad-b995517c5422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.731349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f0488-51b2-45d8-b7ad-b995517c5422 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.732008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f0488-51b2-45d8-b7ad-b995517c5422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.733644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f0488-51b2-45d8-b7ad-b995517c5422 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.756288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1176/2360] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:53.757327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a963c-7615-4c3c-a446-9fb01e47d543 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.770081 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a963c-7615-4c3c-a446-9fb01e47d543 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.775779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225141e-fe4a-4ddb-8e36-ba73874f1db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.776870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225141e-fe4a-4ddb-8e36-ba73874f1db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.777272 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225141e-fe4a-4ddb-8e36-ba73874f1db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.777557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225141e-fe4a-4ddb-8e36-ba73874f1db3 admin admin] PATH_INFO: `/v3/projects/108bfe2bf3ef4249bb1f940b4ddbd8c6/users/a2162bb4321b44e48f56d41ddec1abfb/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.797486 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225141e-fe4a-4ddb-8e36-ba73874f1db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108bfe2bf3ef4249bb1f940b4ddbd8c6, user_id=a2162bb4321b44e48f56d41ddec1abfb, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.802285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225141e-fe4a-4ddb-8e36-ba73874f1db3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.823106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1185/2361] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:53 2022] PUT /identity/v3/projects/108bfe2bf3ef4249bb1f940b4ddbd8c6/users/a2162bb4321b44e48f56d41ddec1abfb/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:53.829087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f0488-51b2-45d8-b7ad-b995517c5422 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.843267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f0488-51b2-45d8-b7ad-b995517c5422 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.910909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3a2bd-00ea-4e51-92d0-ccd1bcf7e527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.911664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3a2bd-00ea-4e51-92d0-ccd1bcf7e527 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.912045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3a2bd-00ea-4e51-92d0-ccd1bcf7e527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.912391 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3a2bd-00ea-4e51-92d0-ccd1bcf7e527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.913523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3a2bd-00ea-4e51-92d0-ccd1bcf7e527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.914967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3a2bd-00ea-4e51-92d0-ccd1bcf7e527 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.928682 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1177/2362] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:53.933659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225141e-fe4a-4ddb-8e36-ba73874f1db3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.944809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225141e-fe4a-4ddb-8e36-ba73874f1db3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.951837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f791b-fea3-4fde-9a25-0470b02ae093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.952526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f791b-fea3-4fde-9a25-0470b02ae093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.952919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f791b-fea3-4fde-9a25-0470b02ae093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.953405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f791b-fea3-4fde-9a25-0470b02ae093 admin admin] PATH_INFO: `/v3/projects/108bfe2bf3ef4249bb1f940b4ddbd8c6/users/a2162bb4321b44e48f56d41ddec1abfb/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.955329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f791b-fea3-4fde-9a25-0470b02ae093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108bfe2bf3ef4249bb1f940b4ddbd8c6, user_id=a2162bb4321b44e48f56d41ddec1abfb, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.958023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f791b-fea3-4fde-9a25-0470b02ae093 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.970290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1186/2363] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:53 2022] PUT /identity/v3/projects/108bfe2bf3ef4249bb1f940b4ddbd8c6/users/a2162bb4321b44e48f56d41ddec1abfb/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:01.498125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeae45-bd59-40f5-98a6-ac0987947e39 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.498652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeae45-bd59-40f5-98a6-ac0987947e39 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.499116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeae45-bd59-40f5-98a6-ac0987947e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.527856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0eeeae45-bd59-40f5-98a6-ac0987947e39 None None] MFA Rules not processed for user `a2162bb4321b44e48f56d41ddec1abfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:01.529097 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0eeeae45-bd59-40f5-98a6-ac0987947e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:01.550695 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:01.607496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1178/2364] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:04:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:01.621550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f791b-fea3-4fde-9a25-0470b02ae093 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:01.623237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-925f791b-fea3-4fde-9a25-0470b02ae093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:01.705035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f791b-fea3-4fde-9a25-0470b02ae093 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:01.711312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124edbd-d5fe-42a8-aaf9-8470f559e815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:01.712086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124edbd-d5fe-42a8-aaf9-8470f559e815 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.712365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124edbd-d5fe-42a8-aaf9-8470f559e815 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.712719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124edbd-d5fe-42a8-aaf9-8470f559e815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.779631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:01.798932 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124edbd-d5fe-42a8-aaf9-8470f559e815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:01.800915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124edbd-d5fe-42a8-aaf9-8470f559e815 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:01.813683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1187/2365] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:04.251343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba1a50-a628-4c22-820a-5cf72b77d308 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:04.252368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba1a50-a628-4c22-820a-5cf72b77d308 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:04.252860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba1a50-a628-4c22-820a-5cf72b77d308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:04.282253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c3ba1a50-a628-4c22-820a-5cf72b77d308 None None] MFA Rules not processed for user `5c95c8ce27fd43c6a6fe10443b6bde13`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:04.283672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c3ba1a50-a628-4c22-820a-5cf72b77d308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:04.377056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1179/2366] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:04:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:11.808217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=7 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:12.776414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124edbd-d5fe-42a8-aaf9-8470f559e815 service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:12.832853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124edbd-d5fe-42a8-aaf9-8470f559e815 service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:12.836640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e4ec2-7379-4989-9193-12ed504037b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:12.837278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e4ec2-7379-4989-9193-12ed504037b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:12.837558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e4ec2-7379-4989-9193-12ed504037b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:12.837830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e4ec2-7379-4989-9193-12ed504037b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:12.840102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e81e4ec2-7379-4989-9193-12ed504037b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:12.902290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e4ec2-7379-4989-9193-12ed504037b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:12.903878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e4ec2-7379-4989-9193-12ed504037b9 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:12.913238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1188/2367] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:14.364206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=46 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:17.043893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba1a50-a628-4c22-820a-5cf72b77d308 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:17.046166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c3ba1a50-a628-4c22-820a-5cf72b77d308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:17.160972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba1a50-a628-4c22-820a-5cf72b77d308 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:17.173140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:17.174670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:17.175188 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:17.175673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:17.178327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:17.194007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:17.196074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:17.209774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1180/2368] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:22.909045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=50 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:24.795133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd7bd9-adb6-476d-8bb1-15e746b314db None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.795727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd7bd9-adb6-476d-8bb1-15e746b314db None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.795727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd7bd9-adb6-476d-8bb1-15e746b314db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.814489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-94cd7bd9-adb6-476d-8bb1-15e746b314db None None] MFA Rules not processed for user `d467895d586a43719d2abdf4105187f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:24.815663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-94cd7bd9-adb6-476d-8bb1-15e746b314db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:24.822421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1189/2369] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:04:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:27.203577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=32 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:34.809182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:42.248019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:42.249475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:42.261184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00618f-a45b-4ce9-a76c-101520c62fc0 service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:42.265023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659e81d-85a8-477f-9bfe-76c0bad16924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:42.265645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659e81d-85a8-477f-9bfe-76c0bad16924 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:42.266136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659e81d-85a8-477f-9bfe-76c0bad16924 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:42.266409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659e81d-85a8-477f-9bfe-76c0bad16924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:42.275998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659e81d-85a8-477f-9bfe-76c0bad16924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:42.277638 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659e81d-85a8-477f-9bfe-76c0bad16924 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:42.286852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1181/2370] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:04:52.282710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=6 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:08.332874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e89111-7a7f-477e-b384-7c6192fcf87f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:08.333464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e89111-7a7f-477e-b384-7c6192fcf87f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:08.333464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e89111-7a7f-477e-b384-7c6192fcf87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:08.360818 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-62e89111-7a7f-477e-b384-7c6192fcf87f None None] MFA Rules not processed for user `cc7cf306aadb45d1b65252b53c86b612`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:08.362054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-62e89111-7a7f-477e-b384-7c6192fcf87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:08.416357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1190/2371] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:05:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:18.408683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=25 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:44.117130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c0733-d2fe-4ee6-a5a7-f09cad5347c4 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:44.117722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c0733-d2fe-4ee6-a5a7-f09cad5347c4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:44.117722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c0733-d2fe-4ee6-a5a7-f09cad5347c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:44.138383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3f8c0733-d2fe-4ee6-a5a7-f09cad5347c4 None None] MFA Rules not processed for user `5c95c8ce27fd43c6a6fe10443b6bde13`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:44.139607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3f8c0733-d2fe-4ee6-a5a7-f09cad5347c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:44.146738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1182/2372] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:05:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:54.132019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:08.972655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e89111-7a7f-477e-b384-7c6192fcf87f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:08.974073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-62e89111-7a7f-477e-b384-7c6192fcf87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:09.030795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e89111-7a7f-477e-b384-7c6192fcf87f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:09.034363 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1676e-276d-4894-8271-812b32ad065f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:09.034979 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1676e-276d-4894-8271-812b32ad065f service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:09.035238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1676e-276d-4894-8271-812b32ad065f service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:09.035783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1676e-276d-4894-8271-812b32ad065f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:09.037552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-07c1676e-276d-4894-8271-812b32ad065f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:09.050738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1676e-276d-4894-8271-812b32ad065f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:09.052257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1676e-276d-4894-8271-812b32ad065f service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:09.063178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1191/2373] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:13.044475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c0733-d2fe-4ee6-a5a7-f09cad5347c4 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:13.054594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c0733-d2fe-4ee6-a5a7-f09cad5347c4 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:13.058117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3664b3-91e9-4477-90eb-d817c32c2958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:13.058704 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3664b3-91e9-4477-90eb-d817c32c2958 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:13.059001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3664b3-91e9-4477-90eb-d817c32c2958 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:13.059275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3664b3-91e9-4477-90eb-d817c32c2958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:13.060982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ac3664b3-91e9-4477-90eb-d817c32c2958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:13.073894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3664b3-91e9-4477-90eb-d817c32c2958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:13.075284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3664b3-91e9-4477-90eb-d817c32c2958 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:13.085963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1183/2374] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:16.662362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1676e-276d-4894-8271-812b32ad065f service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:16.671736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1676e-276d-4894-8271-812b32ad065f service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:16.676192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589b9ad-da01-4c70-8868-64231a6cb3a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:16.676800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b9ad-da01-4c70-8868-64231a6cb3a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:16.677098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b9ad-da01-4c70-8868-64231a6cb3a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:16.677380 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b9ad-da01-4c70-8868-64231a6cb3a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:16.679180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5589b9ad-da01-4c70-8868-64231a6cb3a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:16.756041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589b9ad-da01-4c70-8868-64231a6cb3a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:16.757709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589b9ad-da01-4c70-8868-64231a6cb3a4 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:16.767573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1192/2375] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:21.444239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3664b3-91e9-4477-90eb-d817c32c2958 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:21.458113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3664b3-91e9-4477-90eb-d817c32c2958 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:21.467272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:21.468138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:21.470001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:21.470474 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:21.473549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:21.486725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:21.488512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:21.498599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1184/2376] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:25.549186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf332-cea2-480d-bba4-06f64ac1b8db None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:25.553694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf332-cea2-480d-bba4-06f64ac1b8db None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:25.554147 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf332-cea2-480d-bba4-06f64ac1b8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:25.577114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a2cdf332-cea2-480d-bba4-06f64ac1b8db None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:25.579617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a2cdf332-cea2-480d-bba4-06f64ac1b8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:25.652084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1193/2377] 10.222.0.78 () {58 vars in 967 bytes} [Tue Aug 30 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:06:25.759894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:25.762886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:25.863569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa0065-316b-4a0c-ac04-e59f2e2086cc service nova] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:25.869513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292877-b88a-4707-b376-c5c1807cefe0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:25.870432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292877-b88a-4707-b376-c5c1807cefe0 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:25.870883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292877-b88a-4707-b376-c5c1807cefe0 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:25.871362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292877-b88a-4707-b376-c5c1807cefe0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:25.882094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98292877-b88a-4707-b376-c5c1807cefe0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:25.883951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98292877-b88a-4707-b376-c5c1807cefe0 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:25.914881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1185/2378] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:06:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:06:27.541637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ca074-1f6f-4846-9b34-d8f70ee02030 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:27.542150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ca074-1f6f-4846-9b34-d8f70ee02030 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:27.542150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ca074-1f6f-4846-9b34-d8f70ee02030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:27.563450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f12ca074-1f6f-4846-9b34-d8f70ee02030 None None] MFA Rules not processed for user `c6fe0e0033de447da70be9bc0228eba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:27.564666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f12ca074-1f6f-4846-9b34-d8f70ee02030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:27.574362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1194/2379] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:06:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:06:28.024509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292877-b88a-4707-b376-c5c1807cefe0 service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:28.034583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292877-b88a-4707-b376-c5c1807cefe0 service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:28.038672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:28.039266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:28.039547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:28.039817 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:28.041660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:28.051583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:28.052998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:28.061237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1186/2380] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:28.402158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ca074-1f6f-4846-9b34-d8f70ee02030 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:28.403587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f12ca074-1f6f-4846-9b34-d8f70ee02030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:28.420027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ca074-1f6f-4846-9b34-d8f70ee02030 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:28.426072 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cc1ad-2076-46e8-b65d-4b2f898e7f14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:28.426689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cc1ad-2076-46e8-b65d-4b2f898e7f14 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:28.426969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cc1ad-2076-46e8-b65d-4b2f898e7f14 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:28.427240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cc1ad-2076-46e8-b65d-4b2f898e7f14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:28.429162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-663cc1ad-2076-46e8-b65d-4b2f898e7f14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:28.442429 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cc1ad-2076-46e8-b65d-4b2f898e7f14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:28.444028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cc1ad-2076-46e8-b65d-4b2f898e7f14 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:28.455159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1195/2381] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:30.029970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:30.031801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:30.048448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c57497-1c1e-478f-aab4-83da0c7d95a1 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:30.056104 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0464b56-73a4-4423-a2f0-17b0036dedd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:30.056689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0464b56-73a4-4423-a2f0-17b0036dedd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:30.056964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0464b56-73a4-4423-a2f0-17b0036dedd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:30.057266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0464b56-73a4-4423-a2f0-17b0036dedd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:30.070229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f0464b56-73a4-4423-a2f0-17b0036dedd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:30.083784 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0464b56-73a4-4423-a2f0-17b0036dedd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:30.085917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0464b56-73a4-4423-a2f0-17b0036dedd2 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:30.099535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1187/2382] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:30.970333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f2e15-135d-4b05-9bbd-56165e4eeb26 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:30.971897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f2e15-135d-4b05-9bbd-56165e4eeb26 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:30.972179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f2e15-135d-4b05-9bbd-56165e4eeb26 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:30.974030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1196/2383] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:06:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:30.988120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637baded-4dca-4e3b-8268-604b1bb016f6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:30.988873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637baded-4dca-4e3b-8268-604b1bb016f6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:30.989368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637baded-4dca-4e3b-8268-604b1bb016f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:31.027306 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-637baded-4dca-4e3b-8268-604b1bb016f6 None None] MFA Rules not processed for user `8f860912e1f0406cadd70da440b2ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:31.028591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-637baded-4dca-4e3b-8268-604b1bb016f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:31.056551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1188/2384] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:06:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:06:31.068970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f2e15-135d-4b05-9bbd-56165e4eeb26 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:31.085454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f2e15-135d-4b05-9bbd-56165e4eeb26 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:31.091307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55896d6-b354-45b7-935d-8abc56486be4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:31.091913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55896d6-b354-45b7-935d-8abc56486be4 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:31.092192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55896d6-b354-45b7-935d-8abc56486be4 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:31.092463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55896d6-b354-45b7-935d-8abc56486be4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:31.102682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55896d6-b354-45b7-935d-8abc56486be4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:31.104312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55896d6-b354-45b7-935d-8abc56486be4 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:31.115770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1197/2385] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:31.710178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13561172-507e-46ce-8a30-7c6f20cb7426 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:31.710497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13561172-507e-46ce-8a30-7c6f20cb7426 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:31.711119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13561172-507e-46ce-8a30-7c6f20cb7426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:31.732860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-13561172-507e-46ce-8a30-7c6f20cb7426 None None] MFA Rules not processed for user `cc7cf306aadb45d1b65252b53c86b612`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:31.734889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-13561172-507e-46ce-8a30-7c6f20cb7426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:31.742325 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1189/2386] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:06:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:06:32.202530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55896d6-b354-45b7-935d-8abc56486be4 service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:32.214429 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55896d6-b354-45b7-935d-8abc56486be4 service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:32.220305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6668f2c-6f07-4bde-af58-885af89bc86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:32.221070 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6668f2c-6f07-4bde-af58-885af89bc86f service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:32.221377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6668f2c-6f07-4bde-af58-885af89bc86f service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:32.221653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6668f2c-6f07-4bde-af58-885af89bc86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:32.223487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a6668f2c-6f07-4bde-af58-885af89bc86f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:32.238364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6668f2c-6f07-4bde-af58-885af89bc86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:32.239826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6668f2c-6f07-4bde-af58-885af89bc86f service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:32.244360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:32.252793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1198/2387] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:34.896482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13561172-507e-46ce-8a30-7c6f20cb7426 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:34.898025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-13561172-507e-46ce-8a30-7c6f20cb7426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:34.903205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=63 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:34.974851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13561172-507e-46ce-8a30-7c6f20cb7426 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:34.979665 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:34.980305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:34.980576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:34.980863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] PATH_INFO: `/v3/users/0aa5b198f4e14d52a369248f1e6c0f47` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:34.982007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa5b198f4e14d52a369248f1e6c0f47)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:34.984574 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:35.033763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa5b198f4e14d52a369248f1e6c0f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.041405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa5b198f4e14d52a369248f1e6c0f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.042156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa5b198f4e14d52a369248f1e6c0f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.042801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa5b198f4e14d52a369248f1e6c0f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.055562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa5b198f4e14d52a369248f1e6c0f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.068759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1190/2388] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:06:34 2022] DELETE /identity/v3/users/0aa5b198f4e14d52a369248f1e6c0f47 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:35.143989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6668f2c-6f07-4bde-af58-885af89bc86f service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:35.153297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6668f2c-6f07-4bde-af58-885af89bc86f service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:35.214051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbf8ea-af73-44ac-932a-4e7ede3e7382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:35.214848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbf8ea-af73-44ac-932a-4e7ede3e7382 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:35.215252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbf8ea-af73-44ac-932a-4e7ede3e7382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:35.215613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbf8ea-af73-44ac-932a-4e7ede3e7382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:35.219624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-77dbf8ea-af73-44ac-932a-4e7ede3e7382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:35.231551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbf8ea-af73-44ac-932a-4e7ede3e7382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:35.233169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbf8ea-af73-44ac-932a-4e7ede3e7382 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:35.243733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1199/2389] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:06:35.265404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:35.266848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:35.344840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c067-166d-432b-8862-c0e5eff3fcc8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:35.349642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:35.350269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:35.350546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:35.350822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] PATH_INFO: `/v3/projects/fca43fd867154911badb413c7046befd` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:35.351949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca43fd867154911badb413c7046befd)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:35.353881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:35.383392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:35.414191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca43fd867154911badb413c7046befd', 'request_id': 'req-f11a6766-403e-4f5f-bc06-9c9eaef8042d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.422910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca43fd867154911badb413c7046befd', 'request_id': 'req-f11a6766-403e-4f5f-bc06-9c9eaef8042d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.433506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca43fd867154911badb413c7046befd', 'request_id': 'req-f11a6766-403e-4f5f-bc06-9c9eaef8042d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.445092 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] The token cache is being invalidate because project fca43fd867154911badb413c7046befd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:06:35.445398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.446037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.447725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1191/2390] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:06:35 2022] DELETE /identity/v3/projects/fca43fd867154911badb413c7046befd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:35.451573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbf8ea-af73-44ac-932a-4e7ede3e7382 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:35.477109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbf8ea-af73-44ac-932a-4e7ede3e7382 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:35.549885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:35.550750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:35.551029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:35.551348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] PATH_INFO: `/v3/users/472cc26b48ad4976bcd28395fbeefa1e` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:35.552568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=472cc26b48ad4976bcd28395fbeefa1e)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:35.554718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:35.635077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '472cc26b48ad4976bcd28395fbeefa1e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.646549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '472cc26b48ad4976bcd28395fbeefa1e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.663793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '472cc26b48ad4976bcd28395fbeefa1e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.666149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '472cc26b48ad4976bcd28395fbeefa1e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.715295 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '472cc26b48ad4976bcd28395fbeefa1e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:35.726613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1200/2391] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:06:35 2022] DELETE /identity/v3/users/472cc26b48ad4976bcd28395fbeefa1e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:36.286190 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:36.287907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:36.333134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6766-403e-4f5f-bc06-9c9eaef8042d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:36.371728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:36.374462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:36.414051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:36.476847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:36.477541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:36.477816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:36.478088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] PATH_INFO: `/v3/projects/f7ddbc07b2f149d2867b67c837c60afd` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:36.479244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ddbc07b2f149d2867b67c837c60afd)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:36.481300 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:36.487276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259fef3-114e-407e-ace2-ccd6a26ee735 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:36.498095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de492bf8-d694-46ac-96b9-a638bf405c9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:36.500967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de492bf8-d694-46ac-96b9-a638bf405c9c service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:36.502534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de492bf8-d694-46ac-96b9-a638bf405c9c service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:36.503030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de492bf8-d694-46ac-96b9-a638bf405c9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:36.508075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-de492bf8-d694-46ac-96b9-a638bf405c9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:36.527881 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de492bf8-d694-46ac-96b9-a638bf405c9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:36.530529 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de492bf8-d694-46ac-96b9-a638bf405c9c service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:36.547128 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1201/2392] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:06:36.550066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ddbc07b2f149d2867b67c837c60afd', 'request_id': 'req-ab089722-bf99-4c85-a571-ea54fefb5600'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:36.557514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ddbc07b2f149d2867b67c837c60afd', 'request_id': 'req-ab089722-bf99-4c85-a571-ea54fefb5600'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:36.567713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ddbc07b2f149d2867b67c837c60afd', 'request_id': 'req-ab089722-bf99-4c85-a571-ea54fefb5600'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:36.584186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] The token cache is being invalidate because project f7ddbc07b2f149d2867b67c837c60afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:06:36.584576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:36.585387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:36.597186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1192/2393] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:06:36 2022] DELETE /identity/v3/projects/f7ddbc07b2f149d2867b67c837c60afd => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:46.540154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=24 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:46.578910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:49.022915 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de492bf8-d694-46ac-96b9-a638bf405c9c service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:49.034929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de492bf8-d694-46ac-96b9-a638bf405c9c service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:49.102269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:49.102919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:49.103202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:49.103473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:49.105300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:49.117708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:49.119319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:49.130618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1202/2394] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:52.352337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:52.353863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:52.411126 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab089722-bf99-4c85-a571-ea54fefb5600 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:52.414773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:52.415374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:52.415655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:52.415926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:52.417661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:52.478932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:52.480349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:52.490347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1193/2395] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:56.308169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:56.309849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:56.379414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:56.381562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:56.399274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e79b3-d5ae-493c-9c00-e6a3d8650ab8 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:56.400569 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d58d7f-7190-4b17-b376-aecbe470a08c service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:56.405023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:56.406077 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:56.406454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:56.406816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] PATH_INFO: `/v3/users/5c95c8ce27fd43c6a6fe10443b6bde13` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:56.408193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c95c8ce27fd43c6a6fe10443b6bde13)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:56.409830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0709a4e-d062-40c0-945d-f52e2261658c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:56.411043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0709a4e-d062-40c0-945d-f52e2261658c service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:56.411219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:56.411763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0709a4e-d062-40c0-945d-f52e2261658c service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:56.412217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0709a4e-d062-40c0-945d-f52e2261658c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:56.414392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e0709a4e-d062-40c0-945d-f52e2261658c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:56.427419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0709a4e-d062-40c0-945d-f52e2261658c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:56.429616 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0709a4e-d062-40c0-945d-f52e2261658c service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:56.442730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1203/2396] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:56.475498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c95c8ce27fd43c6a6fe10443b6bde13'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.484120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c95c8ce27fd43c6a6fe10443b6bde13'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.484852 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c95c8ce27fd43c6a6fe10443b6bde13'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.485561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c95c8ce27fd43c6a6fe10443b6bde13'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.499068 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1e9e8a16-b7b9-4ed7-8a3d-01cedaeb5721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c95c8ce27fd43c6a6fe10443b6bde13'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.512258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1194/2397] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:06:56 2022] DELETE /identity/v3/users/5c95c8ce27fd43c6a6fe10443b6bde13 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:56.713345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0709a4e-d062-40c0-945d-f52e2261658c service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:56.725848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0709a4e-d062-40c0-945d-f52e2261658c service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:56.780016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68548a3-8970-4572-afef-eb94f1ca9e47 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:56.780307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68548a3-8970-4572-afef-eb94f1ca9e47 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:56.780583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68548a3-8970-4572-afef-eb94f1ca9e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:56.808191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a68548a3-8970-4572-afef-eb94f1ca9e47 None None] MFA Rules not processed for user `d467895d586a43719d2abdf4105187f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:56.809585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a68548a3-8970-4572-afef-eb94f1ca9e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:56.810899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:56.811620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:56.811971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:56.812338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] PATH_INFO: `/v3/projects/ceebbaa5dd9d4110a6393190c5b466f0` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:56.813777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceebbaa5dd9d4110a6393190c5b466f0)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:56.815716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:56.830584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:56.834579 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:56.869907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceebbaa5dd9d4110a6393190c5b466f0', 'request_id': 'req-5a5df432-d91d-4082-949b-98b3dc3dcaeb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.877120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1195/2398] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:06:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:06:56.878245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceebbaa5dd9d4110a6393190c5b466f0', 'request_id': 'req-5a5df432-d91d-4082-949b-98b3dc3dcaeb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.888014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceebbaa5dd9d4110a6393190c5b466f0', 'request_id': 'req-5a5df432-d91d-4082-949b-98b3dc3dcaeb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.899957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] The token cache is being invalidate because project ceebbaa5dd9d4110a6393190c5b466f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:06:56.900253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.900938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-5a5df432-d91d-4082-949b-98b3dc3dcaeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.905310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1204/2399] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:06:56 2022] DELETE /identity/v3/projects/ceebbaa5dd9d4110a6393190c5b466f0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:56.907906 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0ad19-6815-43f2-9f46-06840dcc5f86 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:56.908187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0ad19-6815-43f2-9f46-06840dcc5f86 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:56.908889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0ad19-6815-43f2-9f46-06840dcc5f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:56.932195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e2c0ad19-6815-43f2-9f46-06840dcc5f86 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:56.933449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e2c0ad19-6815-43f2-9f46-06840dcc5f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:57.003108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1196/2400] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:06:57.009835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d3e9-da7e-4312-9d1b-d89f8dc327b3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.010122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d3e9-da7e-4312-9d1b-d89f8dc327b3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.010462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d3e9-da7e-4312-9d1b-d89f8dc327b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.034716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5a19d3e9-da7e-4312-9d1b-d89f8dc327b3 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:57.035956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5a19d3e9-da7e-4312-9d1b-d89f8dc327b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:57.044428 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1205/2401] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:06:57.049626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0ad19-6815-43f2-9f46-06840dcc5f86 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:57.062330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0ad19-6815-43f2-9f46-06840dcc5f86 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:57.067121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa204f-4394-433a-a55d-f43db952b2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:57.067730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa204f-4394-433a-a55d-f43db952b2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.068005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa204f-4394-433a-a55d-f43db952b2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.068298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa204f-4394-433a-a55d-f43db952b2ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.068947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa204f-4394-433a-a55d-f43db952b2ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:57.069315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa204f-4394-433a-a55d-f43db952b2ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:57.070929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa204f-4394-433a-a55d-f43db952b2ea admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:57.083967 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1197/2402] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:57.088516 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19d3e9-da7e-4312-9d1b-d89f8dc327b3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:57.098817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19d3e9-da7e-4312-9d1b-d89f8dc327b3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:57.105404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506e1b5-85ad-4734-8455-fa7a5bde5934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:57.105934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506e1b5-85ad-4734-8455-fa7a5bde5934 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.106330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506e1b5-85ad-4734-8455-fa7a5bde5934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.106724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506e1b5-85ad-4734-8455-fa7a5bde5934 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.108580 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506e1b5-85ad-4734-8455-fa7a5bde5934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:57.110833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506e1b5-85ad-4734-8455-fa7a5bde5934 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:57.153901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa204f-4394-433a-a55d-f43db952b2ea admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:57.161115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1206/2403] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:06:57 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:06:57.165894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa204f-4394-433a-a55d-f43db952b2ea admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:57.229523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621c9e0-8aca-4c75-9411-c66e5c7ff80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:57.230125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621c9e0-8aca-4c75-9411-c66e5c7ff80b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.230399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621c9e0-8aca-4c75-9411-c66e5c7ff80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.230665 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621c9e0-8aca-4c75-9411-c66e5c7ff80b admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.231423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621c9e0-8aca-4c75-9411-c66e5c7ff80b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:57.233894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621c9e0-8aca-4c75-9411-c66e5c7ff80b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:57.260708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1198/2404] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:06:57 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:06:57.261955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506e1b5-85ad-4734-8455-fa7a5bde5934 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:57.274651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506e1b5-85ad-4734-8455-fa7a5bde5934 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:57.279953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d744f-6409-4a0b-a583-2b9a87abaf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:57.280547 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d744f-6409-4a0b-a583-2b9a87abaf12 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.280823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d744f-6409-4a0b-a583-2b9a87abaf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.281111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d744f-6409-4a0b-a583-2b9a87abaf12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.281758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d744f-6409-4a0b-a583-2b9a87abaf12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:57.283334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d744f-6409-4a0b-a583-2b9a87abaf12 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:57.295168 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1207/2405] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:06:57.299299 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621c9e0-8aca-4c75-9411-c66e5c7ff80b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:57.309353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621c9e0-8aca-4c75-9411-c66e5c7ff80b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:57.314141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571494d-a159-400e-9072-2c6e469f46fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:57.314738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571494d-a159-400e-9072-2c6e469f46fe admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.315016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571494d-a159-400e-9072-2c6e469f46fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.315289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571494d-a159-400e-9072-2c6e469f46fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.315960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571494d-a159-400e-9072-2c6e469f46fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:57.317581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571494d-a159-400e-9072-2c6e469f46fe admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:57.333950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1199/2406] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:06:57.337927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d744f-6409-4a0b-a583-2b9a87abaf12 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:57.349543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d744f-6409-4a0b-a583-2b9a87abaf12 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:57.354277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ae547-2b5d-410a-864b-543b8e3c9359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:57.354923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ae547-2b5d-410a-864b-543b8e3c9359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.355204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ae547-2b5d-410a-864b-543b8e3c9359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.355525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ae547-2b5d-410a-864b-543b8e3c9359 admin admin] PATH_INFO: `/v3/projects/54768b167b1c4fefb901f640203e1c5d/users/3ff0c4a8fd8d44bda56ceeb47cd49abf/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.371496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ae547-2b5d-410a-864b-543b8e3c9359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54768b167b1c4fefb901f640203e1c5d, user_id=3ff0c4a8fd8d44bda56ceeb47cd49abf, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:57.374308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ae547-2b5d-410a-864b-543b8e3c9359 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:57.388550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1208/2407] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:06:57 2022] PUT /identity/v3/projects/54768b167b1c4fefb901f640203e1c5d/users/3ff0c4a8fd8d44bda56ceeb47cd49abf/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:57.398975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633900b4-96cf-4c0f-af4b-b3a58545766e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.399582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633900b4-96cf-4c0f-af4b-b3a58545766e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.399582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633900b4-96cf-4c0f-af4b-b3a58545766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.419374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-633900b4-96cf-4c0f-af4b-b3a58545766e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:57.421300 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-633900b4-96cf-4c0f-af4b-b3a58545766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:57.451579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:57.484973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1200/2408] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:06:57.494586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ae547-2b5d-410a-864b-543b8e3c9359 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:57.496280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-299ae547-2b5d-410a-864b-543b8e3c9359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:57.555708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ae547-2b5d-410a-864b-543b8e3c9359 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:57.559404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee0298-7876-412f-8cb4-9decfb574d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:57.560592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee0298-7876-412f-8cb4-9decfb574d39 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.560939 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee0298-7876-412f-8cb4-9decfb574d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.561305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee0298-7876-412f-8cb4-9decfb574d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.570800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee0298-7876-412f-8cb4-9decfb574d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:57.572451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee0298-7876-412f-8cb4-9decfb574d39 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:57.583504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1209/2409] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:02.285410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3e060-a97e-4cf2-94fe-17a88a7a2209 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:02.285976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3e060-a97e-4cf2-94fe-17a88a7a2209 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:02.285976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3e060-a97e-4cf2-94fe-17a88a7a2209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:02.335854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b3d3e060-a97e-4cf2-94fe-17a88a7a2209 None None] MFA Rules not processed for user `3ff0c4a8fd8d44bda56ceeb47cd49abf`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:02.337272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b3d3e060-a97e-4cf2-94fe-17a88a7a2209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:02.422656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1201/2410] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:07:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:02.437788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee0298-7876-412f-8cb4-9decfb574d39 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:02.439377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-59ee0298-7876-412f-8cb4-9decfb574d39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:02.504586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:02.535116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee0298-7876-412f-8cb4-9decfb574d39 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:02.540727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32f740-a4dc-4983-a56a-06424802974b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:02.541380 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32f740-a4dc-4983-a56a-06424802974b service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:02.541655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32f740-a4dc-4983-a56a-06424802974b service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:02.541927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32f740-a4dc-4983-a56a-06424802974b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:02.551195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32f740-a4dc-4983-a56a-06424802974b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:02.552803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32f740-a4dc-4983-a56a-06424802974b service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:02.562499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1210/2411] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:03.940757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907ca8f-2d96-4a74-97eb-733e542292ec None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:03.941268 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907ca8f-2d96-4a74-97eb-733e542292ec None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:03.941353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907ca8f-2d96-4a74-97eb-733e542292ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:03.961173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2907ca8f-2d96-4a74-97eb-733e542292ec None None] MFA Rules not processed for user `3ff0c4a8fd8d44bda56ceeb47cd49abf`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:03.962320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2907ca8f-2d96-4a74-97eb-733e542292ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:03.969955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1202/2412] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:07:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:10.199438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32f740-a4dc-4983-a56a-06424802974b service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:10.214524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32f740-a4dc-4983-a56a-06424802974b service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:10.221442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:10.222145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:10.222485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:10.222824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:10.225018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:10.240610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:10.242278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:10.258159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1211/2413] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:13.955531 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=47 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:20.249691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=20 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:51.895219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907ca8f-2d96-4a74-97eb-733e542292ec None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:51.905212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907ca8f-2d96-4a74-97eb-733e542292ec None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:51.909245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba3317-4516-44d9-893e-0afa54fb4039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:51.909837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba3317-4516-44d9-893e-0afa54fb4039 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:51.910111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba3317-4516-44d9-893e-0afa54fb4039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:51.910375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba3317-4516-44d9-893e-0afa54fb4039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:51.912047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-77ba3317-4516-44d9-893e-0afa54fb4039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:51.924176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba3317-4516-44d9-893e-0afa54fb4039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:51.925617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba3317-4516-44d9-893e-0afa54fb4039 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:51.935406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1203/2414] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:55.187458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:55.189793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:55.205419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a9feb-0358-4494-a38d-d956b228cfb7 service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:55.210303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:55.210925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:55.211207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:55.211484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] PATH_INFO: `/v3/users/cc7cf306aadb45d1b65252b53c86b612` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:55.212759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7cf306aadb45d1b65252b53c86b612)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:55.214876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:55.266364 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7cf306aadb45d1b65252b53c86b612'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.274636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7cf306aadb45d1b65252b53c86b612'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.282595 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7cf306aadb45d1b65252b53c86b612'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.283283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7cf306aadb45d1b65252b53c86b612'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.296605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-236c3db8-fdb0-4fb1-a8b7-75d94c3f0540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7cf306aadb45d1b65252b53c86b612'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.302909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1212/2415] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:55 2022] DELETE /identity/v3/users/cc7cf306aadb45d1b65252b53c86b612 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:55.487088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba3317-4516-44d9-893e-0afa54fb4039 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:55.500298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba3317-4516-44d9-893e-0afa54fb4039 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:55.569794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:55.570454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:55.570787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:55.571060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] PATH_INFO: `/v3/projects/0596692dbfb149139a1e72159c6f93db` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:55.572183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0596692dbfb149139a1e72159c6f93db)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:55.574434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:55.624779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0596692dbfb149139a1e72159c6f93db', 'request_id': 'req-c1831bbd-8596-4379-b3fa-e276b9bb37dd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.630905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0596692dbfb149139a1e72159c6f93db', 'request_id': 'req-c1831bbd-8596-4379-b3fa-e276b9bb37dd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.640199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0596692dbfb149139a1e72159c6f93db', 'request_id': 'req-c1831bbd-8596-4379-b3fa-e276b9bb37dd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.652183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] The token cache is being invalidate because project 0596692dbfb149139a1e72159c6f93db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:55.652468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.653146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c1831bbd-8596-4379-b3fa-e276b9bb37dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:55.654862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1204/2416] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:55 2022] DELETE /identity/v3/projects/0596692dbfb149139a1e72159c6f93db => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:55.660085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a5f3d-ac6d-4db1-8162-422257e792dd None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:55.660403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a5f3d-ac6d-4db1-8162-422257e792dd None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:55.660684 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a5f3d-ac6d-4db1-8162-422257e792dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:55.680946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4d9a5f3d-ac6d-4db1-8162-422257e792dd None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:55.682225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4d9a5f3d-ac6d-4db1-8162-422257e792dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:55.745957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1213/2417] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:55.755478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e942e-b587-4349-b369-edfe882f286f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:55.756053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e942e-b587-4349-b369-edfe882f286f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:55.756492 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e942e-b587-4349-b369-edfe882f286f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:55.776832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e33e942e-b587-4349-b369-edfe882f286f None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:55.778086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e33e942e-b587-4349-b369-edfe882f286f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:55.788359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1205/2418] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:55.792983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a5f3d-ac6d-4db1-8162-422257e792dd None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:55.803975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a5f3d-ac6d-4db1-8162-422257e792dd None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:55.808608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de0d2b-4bb5-42d5-ad0c-cb55d0f5529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:55.809174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de0d2b-4bb5-42d5-ad0c-cb55d0f5529e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:55.809456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de0d2b-4bb5-42d5-ad0c-cb55d0f5529e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:55.809727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de0d2b-4bb5-42d5-ad0c-cb55d0f5529e admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:55.811515 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de0d2b-4bb5-42d5-ad0c-cb55d0f5529e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:55.811803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de0d2b-4bb5-42d5-ad0c-cb55d0f5529e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:55.813435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de0d2b-4bb5-42d5-ad0c-cb55d0f5529e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:55.825004 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1214/2419] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:55.830203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e942e-b587-4349-b369-edfe882f286f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:55.839525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e942e-b587-4349-b369-edfe882f286f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:55.844901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202250-3914-4710-9298-867200b8ce54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:55.845517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202250-3914-4710-9298-867200b8ce54 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:55.845793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202250-3914-4710-9298-867200b8ce54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:55.846058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202250-3914-4710-9298-867200b8ce54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:55.846712 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9202250-3914-4710-9298-867200b8ce54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:55.848608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9202250-3914-4710-9298-867200b8ce54 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:55.876153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1206/2420] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:55 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:07:55.880357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de0d2b-4bb5-42d5-ad0c-cb55d0f5529e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:55.892418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de0d2b-4bb5-42d5-ad0c-cb55d0f5529e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:55.948636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f591aa-c29d-40fa-aeb1-de1ea5c7f5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:55.949282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f591aa-c29d-40fa-aeb1-de1ea5c7f5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:55.949557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f591aa-c29d-40fa-aeb1-de1ea5c7f5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:55.949823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f591aa-c29d-40fa-aeb1-de1ea5c7f5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:55.950954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f591aa-c29d-40fa-aeb1-de1ea5c7f5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:55.952877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f591aa-c29d-40fa-aeb1-de1ea5c7f5c4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:55.960694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:55.977256 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1215/2421] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:55 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:55.981172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202250-3914-4710-9298-867200b8ce54 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:55.991792 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202250-3914-4710-9298-867200b8ce54 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:55.996506 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33699dc-0a56-4579-813b-729420775343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:55.997142 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33699dc-0a56-4579-813b-729420775343 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:55.997523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33699dc-0a56-4579-813b-729420775343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:55.997948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33699dc-0a56-4579-813b-729420775343 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:55.998590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33699dc-0a56-4579-813b-729420775343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:56.000178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33699dc-0a56-4579-813b-729420775343 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:56.010100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1207/2422] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:56.014461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f591aa-c29d-40fa-aeb1-de1ea5c7f5c4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:56.024379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f591aa-c29d-40fa-aeb1-de1ea5c7f5c4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:56.029725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8e34b-48d9-40b2-8d97-300c1da5dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:56.030351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8e34b-48d9-40b2-8d97-300c1da5dee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:56.030627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8e34b-48d9-40b2-8d97-300c1da5dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:56.031142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8e34b-48d9-40b2-8d97-300c1da5dee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:56.031779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8e34b-48d9-40b2-8d97-300c1da5dee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:56.033401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8e34b-48d9-40b2-8d97-300c1da5dee3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:56.043832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1216/2423] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:56.048453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33699dc-0a56-4579-813b-729420775343 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:56.058396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33699dc-0a56-4579-813b-729420775343 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:56.063856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903ebf5-ad33-4e66-9442-fefde84fb846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:56.064430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903ebf5-ad33-4e66-9442-fefde84fb846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:56.064713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903ebf5-ad33-4e66-9442-fefde84fb846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:56.064980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903ebf5-ad33-4e66-9442-fefde84fb846 admin admin] PATH_INFO: `/v3/projects/f2099dd794c54eaa8e3d5eae0f7c355b/users/ece79a4975bb450387cc7f263b719439/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:56.080052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903ebf5-ad33-4e66-9442-fefde84fb846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2099dd794c54eaa8e3d5eae0f7c355b, user_id=ece79a4975bb450387cc7f263b719439, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:56.082727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903ebf5-ad33-4e66-9442-fefde84fb846 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:56.096588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1208/2424] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:56 2022] PUT /identity/v3/projects/f2099dd794c54eaa8e3d5eae0f7c355b/users/ece79a4975bb450387cc7f263b719439/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:56.106780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73614b06-8897-418a-bd6f-31144433dff2 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:56.107725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73614b06-8897-418a-bd6f-31144433dff2 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:56.107725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73614b06-8897-418a-bd6f-31144433dff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:56.134751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-73614b06-8897-418a-bd6f-31144433dff2 None None] MFA Rules not processed for user `ece79a4975bb450387cc7f263b719439`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:56.135938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-73614b06-8897-418a-bd6f-31144433dff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:56.197995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1217/2425] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:07:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:56.211037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903ebf5-ad33-4e66-9442-fefde84fb846 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:56.212791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4903ebf5-ad33-4e66-9442-fefde84fb846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:56.227922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:56.280119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903ebf5-ad33-4e66-9442-fefde84fb846 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:56.284063 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d6e9a-65c2-4ff8-bed0-f82cf02cca9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:56.284668 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d6e9a-65c2-4ff8-bed0-f82cf02cca9c service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:56.284969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d6e9a-65c2-4ff8-bed0-f82cf02cca9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:56.285248 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d6e9a-65c2-4ff8-bed0-f82cf02cca9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:56.293915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d6e9a-65c2-4ff8-bed0-f82cf02cca9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:56.295916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d6e9a-65c2-4ff8-bed0-f82cf02cca9c service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:56.305432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1209/2426] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:04.589601 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73614b06-8897-418a-bd6f-31144433dff2 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:04.591055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-73614b06-8897-418a-bd6f-31144433dff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:04.649899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73614b06-8897-418a-bd6f-31144433dff2 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:04.659780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a35b4d-7765-4f80-9a7d-85d8c2d2aa61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:04.660404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a35b4d-7765-4f80-9a7d-85d8c2d2aa61 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:04.660679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a35b4d-7765-4f80-9a7d-85d8c2d2aa61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:04.660946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a35b4d-7765-4f80-9a7d-85d8c2d2aa61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:04.662872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a2a35b4d-7765-4f80-9a7d-85d8c2d2aa61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:04.729990 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a35b4d-7765-4f80-9a7d-85d8c2d2aa61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:04.731430 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a35b4d-7765-4f80-9a7d-85d8c2d2aa61 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:04.741726 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1218/2427] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:04.849390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d6e9a-65c2-4ff8-bed0-f82cf02cca9c service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:04.860831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d6e9a-65c2-4ff8-bed0-f82cf02cca9c service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:04.865320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:04.865928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:04.866203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:04.866469 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] PATH_INFO: `/v3/users/ece79a4975bb450387cc7f263b719439` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:04.867570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece79a4975bb450387cc7f263b719439)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:04.869528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:04.917013 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece79a4975bb450387cc7f263b719439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:04.926449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece79a4975bb450387cc7f263b719439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:04.927264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece79a4975bb450387cc7f263b719439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:04.927862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece79a4975bb450387cc7f263b719439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:04.945365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece79a4975bb450387cc7f263b719439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:04.964243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1210/2428] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:04 2022] DELETE /identity/v3/users/ece79a4975bb450387cc7f263b719439 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:04.969182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18545-294c-41d0-b1f8-4ac86aea9492 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:04.969499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18545-294c-41d0-b1f8-4ac86aea9492 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:04.969744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18545-294c-41d0-b1f8-4ac86aea9492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:04.989397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-90c18545-294c-41d0-b1f8-4ac86aea9492 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:04.990645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-90c18545-294c-41d0-b1f8-4ac86aea9492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:05.003273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:05.053065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1219/2429] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:05.063713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:05.065291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:05.126170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee45de-8f01-43b3-9e2d-04d61548288d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:05.129884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab8570-b0d9-4b16-84d4-b52a3b3ca1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:05.130518 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab8570-b0d9-4b16-84d4-b52a3b3ca1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.130803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab8570-b0d9-4b16-84d4-b52a3b3ca1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.131070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab8570-b0d9-4b16-84d4-b52a3b3ca1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.141007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab8570-b0d9-4b16-84d4-b52a3b3ca1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:05.142509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab8570-b0d9-4b16-84d4-b52a3b3ca1b6 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:05.152444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1211/2430] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:05.568718 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18545-294c-41d0-b1f8-4ac86aea9492 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:05.570163 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-90c18545-294c-41d0-b1f8-4ac86aea9492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:05.582387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18545-294c-41d0-b1f8-4ac86aea9492 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:05.586985 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:05.587602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.587872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.588139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] PATH_INFO: `/v3/projects/f2099dd794c54eaa8e3d5eae0f7c355b` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.589296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2099dd794c54eaa8e3d5eae0f7c355b)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:05.591180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:05.658353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2099dd794c54eaa8e3d5eae0f7c355b', 'request_id': 'req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:05.664595 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2099dd794c54eaa8e3d5eae0f7c355b', 'request_id': 'req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:05.672839 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2099dd794c54eaa8e3d5eae0f7c355b', 'request_id': 'req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:05.683255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] The token cache is being invalidate because project f2099dd794c54eaa8e3d5eae0f7c355b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:05.683540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:05.684162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-78608ef5-f01a-41e2-95ac-690d2fcaa9e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:05.686444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1220/2431] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:05 2022] DELETE /identity/v3/projects/f2099dd794c54eaa8e3d5eae0f7c355b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:05.691392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9412dd-8400-4962-8ff3-a85a8c8573a8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.691671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9412dd-8400-4962-8ff3-a85a8c8573a8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.691947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9412dd-8400-4962-8ff3-a85a8c8573a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.712831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7e9412dd-8400-4962-8ff3-a85a8c8573a8 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:05.714058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7e9412dd-8400-4962-8ff3-a85a8c8573a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:05.753889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:05.812012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1212/2432] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:05.821110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43b89a-561d-4729-923e-1f3a0c7e074b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.821399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43b89a-561d-4729-923e-1f3a0c7e074b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.821671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43b89a-561d-4729-923e-1f3a0c7e074b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.846664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1f43b89a-561d-4729-923e-1f3a0c7e074b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:05.848238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1f43b89a-561d-4729-923e-1f3a0c7e074b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:05.859589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1221/2433] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:05.864107 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9412dd-8400-4962-8ff3-a85a8c8573a8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:05.875858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9412dd-8400-4962-8ff3-a85a8c8573a8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:05.881228 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c8308-53a2-4a4b-b9b8-730c2d00bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:05.882338 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c8308-53a2-4a4b-b9b8-730c2d00bbdf admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.882810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c8308-53a2-4a4b-b9b8-730c2d00bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.883888 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c8308-53a2-4a4b-b9b8-730c2d00bbdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.885046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c8308-53a2-4a4b-b9b8-730c2d00bbdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:05.885597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c8308-53a2-4a4b-b9b8-730c2d00bbdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:05.887395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c8308-53a2-4a4b-b9b8-730c2d00bbdf admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:05.909893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1213/2434] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:05.911521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43b89a-561d-4729-923e-1f3a0c7e074b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:05.927609 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43b89a-561d-4729-923e-1f3a0c7e074b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:05.940329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3652fb-6687-4c5a-a728-ba343b0870cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:05.940960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3652fb-6687-4c5a-a728-ba343b0870cc admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.941507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3652fb-6687-4c5a-a728-ba343b0870cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.943067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3652fb-6687-4c5a-a728-ba343b0870cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.943067 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3652fb-6687-4c5a-a728-ba343b0870cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:05.944475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3652fb-6687-4c5a-a728-ba343b0870cc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:05.989844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1222/2435] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:05 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:05.990040 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c8308-53a2-4a4b-b9b8-730c2d00bbdf admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:05.994051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c8308-53a2-4a4b-b9b8-730c2d00bbdf admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.051761 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcf0d4-f7e0-4985-b3e4-96f6e6f9d518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.052371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcf0d4-f7e0-4985-b3e4-96f6e6f9d518 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.052647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcf0d4-f7e0-4985-b3e4-96f6e6f9d518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.052932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcf0d4-f7e0-4985-b3e4-96f6e6f9d518 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.053573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcf0d4-f7e0-4985-b3e4-96f6e6f9d518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.055496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcf0d4-f7e0-4985-b3e4-96f6e6f9d518 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.077706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1214/2436] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:05 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:06.083611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3652fb-6687-4c5a-a728-ba343b0870cc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.094507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3652fb-6687-4c5a-a728-ba343b0870cc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.099343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef6ed-ed1b-41f4-8d4f-d173bef89ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.099948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef6ed-ed1b-41f4-8d4f-d173bef89ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.100224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef6ed-ed1b-41f4-8d4f-d173bef89ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.100497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef6ed-ed1b-41f4-8d4f-d173bef89ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.101164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eef6ed-ed1b-41f4-8d4f-d173bef89ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.102768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eef6ed-ed1b-41f4-8d4f-d173bef89ab2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.113159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1223/2437] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:08:06.117438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcf0d4-f7e0-4985-b3e4-96f6e6f9d518 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.128142 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcf0d4-f7e0-4985-b3e4-96f6e6f9d518 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.132865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0a478-a2a1-44e5-84f5-414c4452422e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.133429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0a478-a2a1-44e5-84f5-414c4452422e admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.133711 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0a478-a2a1-44e5-84f5-414c4452422e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.133984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0a478-a2a1-44e5-84f5-414c4452422e admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.134634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0a478-a2a1-44e5-84f5-414c4452422e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.136224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0a478-a2a1-44e5-84f5-414c4452422e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.147782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1215/2438] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:06.152040 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef6ed-ed1b-41f4-8d4f-d173bef89ab2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.163344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef6ed-ed1b-41f4-8d4f-d173bef89ab2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.168042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b5ec-4f8d-4a15-9167-011f629c40aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.168632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b5ec-4f8d-4a15-9167-011f629c40aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.168912 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b5ec-4f8d-4a15-9167-011f629c40aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.169210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113b5ec-4f8d-4a15-9167-011f629c40aa admin admin] PATH_INFO: `/v3/projects/12e6304ad08b4ebcaa9218f84fc191e0/users/eaed78fa27894d4f9ccc5375d8bc17a6/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.189884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113b5ec-4f8d-4a15-9167-011f629c40aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e6304ad08b4ebcaa9218f84fc191e0, user_id=eaed78fa27894d4f9ccc5375d8bc17a6, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.194772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113b5ec-4f8d-4a15-9167-011f629c40aa admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.209668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1224/2439] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:06 2022] PUT /identity/v3/projects/12e6304ad08b4ebcaa9218f84fc191e0/users/eaed78fa27894d4f9ccc5375d8bc17a6/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:06.220530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155165-f38e-4071-9cf4-3956bbc83639 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.220810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155165-f38e-4071-9cf4-3956bbc83639 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.221094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155165-f38e-4071-9cf4-3956bbc83639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.264183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5c155165-f38e-4071-9cf4-3956bbc83639 None None] MFA Rules not processed for user `eaed78fa27894d4f9ccc5375d8bc17a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:06.265586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5c155165-f38e-4071-9cf4-3956bbc83639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:06.361031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1216/2440] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:06.365578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b5ec-4f8d-4a15-9167-011f629c40aa admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.413241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:06.438920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113b5ec-4f8d-4a15-9167-011f629c40aa admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.443804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72837f8f-3cdd-461f-9c2a-94c6a8e48988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.444412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837f8f-3cdd-461f-9c2a-94c6a8e48988 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.444698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837f8f-3cdd-461f-9c2a-94c6a8e48988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.444969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837f8f-3cdd-461f-9c2a-94c6a8e48988 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.445636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72837f8f-3cdd-461f-9c2a-94c6a8e48988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.447550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72837f8f-3cdd-461f-9c2a-94c6a8e48988 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.481265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1225/2441] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:06 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:06.483193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155165-f38e-4071-9cf4-3956bbc83639 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.494762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155165-f38e-4071-9cf4-3956bbc83639 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.525369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:06.569707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1eed0-21be-4e60-aacd-17764975709d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.570312 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1eed0-21be-4e60-aacd-17764975709d admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.570585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1eed0-21be-4e60-aacd-17764975709d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.570849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1eed0-21be-4e60-aacd-17764975709d admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.571472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1eed0-21be-4e60-aacd-17764975709d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.573392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1eed0-21be-4e60-aacd-17764975709d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.598366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1217/2442] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:06 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:06.601981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72837f8f-3cdd-461f-9c2a-94c6a8e48988 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.613681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72837f8f-3cdd-461f-9c2a-94c6a8e48988 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.619253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02231301-55d6-44fc-95bb-3cbd273056e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.619975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02231301-55d6-44fc-95bb-3cbd273056e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.620356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02231301-55d6-44fc-95bb-3cbd273056e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.620724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02231301-55d6-44fc-95bb-3cbd273056e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.621496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02231301-55d6-44fc-95bb-3cbd273056e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.623867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02231301-55d6-44fc-95bb-3cbd273056e9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.638401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1226/2443] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:08:06.643485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1eed0-21be-4e60-aacd-17764975709d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.655902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1eed0-21be-4e60-aacd-17764975709d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.661212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b419563-9fcd-4f21-973a-25c3e7267cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.662133 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419563-9fcd-4f21-973a-25c3e7267cd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.662536 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419563-9fcd-4f21-973a-25c3e7267cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.662913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419563-9fcd-4f21-973a-25c3e7267cd7 admin admin] PATH_INFO: `/v3/projects/f1a9dfae25d545cbb0ffc14973ea2f8b/users/0c0c4a6733c048cda59a80e351995463/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.683616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b419563-9fcd-4f21-973a-25c3e7267cd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a9dfae25d545cbb0ffc14973ea2f8b, user_id=0c0c4a6733c048cda59a80e351995463, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.686860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b419563-9fcd-4f21-973a-25c3e7267cd7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.706932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1218/2444] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:06 2022] PUT /identity/v3/projects/f1a9dfae25d545cbb0ffc14973ea2f8b/users/0c0c4a6733c048cda59a80e351995463/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:06.708367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02231301-55d6-44fc-95bb-3cbd273056e9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.722803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02231301-55d6-44fc-95bb-3cbd273056e9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.796334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7593215-0165-4183-8790-d638c67c5cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.797467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7593215-0165-4183-8790-d638c67c5cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.797841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7593215-0165-4183-8790-d638c67c5cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.798264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7593215-0165-4183-8790-d638c67c5cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.799197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7593215-0165-4183-8790-d638c67c5cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.800887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7593215-0165-4183-8790-d638c67c5cc9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.817439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1227/2445] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:06.818588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b419563-9fcd-4f21-973a-25c3e7267cd7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.830446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b419563-9fcd-4f21-973a-25c3e7267cd7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.843910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ed1b1-4117-448e-8ea1-f12f4c3701bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.844727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ed1b1-4117-448e-8ea1-f12f4c3701bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.845117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ed1b1-4117-448e-8ea1-f12f4c3701bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.845491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ed1b1-4117-448e-8ea1-f12f4c3701bd admin admin] PATH_INFO: `/v3/projects/f1a9dfae25d545cbb0ffc14973ea2f8b/users/0c0c4a6733c048cda59a80e351995463/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.847571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ed1b1-4117-448e-8ea1-f12f4c3701bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a9dfae25d545cbb0ffc14973ea2f8b, user_id=0c0c4a6733c048cda59a80e351995463, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.850334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ed1b1-4117-448e-8ea1-f12f4c3701bd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.862558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1219/2446] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:06 2022] PUT /identity/v3/projects/f1a9dfae25d545cbb0ffc14973ea2f8b/users/0c0c4a6733c048cda59a80e351995463/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:06.873860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a58d9-dd0f-4eb6-8e53-fb38227b4ffe None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.874145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a58d9-dd0f-4eb6-8e53-fb38227b4ffe None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.874414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a58d9-dd0f-4eb6-8e53-fb38227b4ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.907613 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-479a58d9-dd0f-4eb6-8e53-fb38227b4ffe None None] MFA Rules not processed for user `0c0c4a6733c048cda59a80e351995463`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:06.908392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-479a58d9-dd0f-4eb6-8e53-fb38227b4ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:06.976749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1228/2447] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:06.988703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ed1b1-4117-448e-8ea1-f12f4c3701bd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.990450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-be1ed1b1-4117-448e-8ea1-f12f4c3701bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:07.054613 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ed1b1-4117-448e-8ea1-f12f4c3701bd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:07.058830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42b758-54f8-4ec2-bdee-977cd99c77e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:07.059599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42b758-54f8-4ec2-bdee-977cd99c77e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:07.059963 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42b758-54f8-4ec2-bdee-977cd99c77e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:07.060301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42b758-54f8-4ec2-bdee-977cd99c77e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:07.070454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42b758-54f8-4ec2-bdee-977cd99c77e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:07.071864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42b758-54f8-4ec2-bdee-977cd99c77e6 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:07.082014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1220/2448] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:07.164960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a58d9-dd0f-4eb6-8e53-fb38227b4ffe None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:07.177928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a58d9-dd0f-4eb6-8e53-fb38227b4ffe None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:07.182525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567be71-2dd2-4c54-b5a3-9de8cea6a8af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:07.183225 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567be71-2dd2-4c54-b5a3-9de8cea6a8af service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:07.183630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567be71-2dd2-4c54-b5a3-9de8cea6a8af service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:07.183962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567be71-2dd2-4c54-b5a3-9de8cea6a8af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:07.229943 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:07.279018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567be71-2dd2-4c54-b5a3-9de8cea6a8af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:07.288478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567be71-2dd2-4c54-b5a3-9de8cea6a8af service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:07.299357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1229/2449] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:08:08.586790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42b758-54f8-4ec2-bdee-977cd99c77e6 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:08.589634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7d42b758-54f8-4ec2-bdee-977cd99c77e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:08.663592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42b758-54f8-4ec2-bdee-977cd99c77e6 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:08.668820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:08.669477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:08.669812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:08.670113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] PATH_INFO: `/v3/users/c6fe0e0033de447da70be9bc0228eba2` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:08.671236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6fe0e0033de447da70be9bc0228eba2)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:08.673221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:08.699200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:08.725239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6fe0e0033de447da70be9bc0228eba2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.734773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6fe0e0033de447da70be9bc0228eba2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.735646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6fe0e0033de447da70be9bc0228eba2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.736283 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6fe0e0033de447da70be9bc0228eba2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.750864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-54e6b7c9-87fb-4480-b4b2-7a907d7e03b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6fe0e0033de447da70be9bc0228eba2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.765596 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1221/2450] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:08 2022] DELETE /identity/v3/users/c6fe0e0033de447da70be9bc0228eba2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:08.983046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567be71-2dd2-4c54-b5a3-9de8cea6a8af service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:08.999276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567be71-2dd2-4c54-b5a3-9de8cea6a8af service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.064160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.064919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.065231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.065501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] PATH_INFO: `/v3/projects/f40b3e29a18b48cba6800c9f3a0dfee9` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.066690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40b3e29a18b48cba6800c9f3a0dfee9)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.068695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.123373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40b3e29a18b48cba6800c9f3a0dfee9', 'request_id': 'req-1efe0a00-8e3f-4b08-a701-7f429533e404'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.130537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40b3e29a18b48cba6800c9f3a0dfee9', 'request_id': 'req-1efe0a00-8e3f-4b08-a701-7f429533e404'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.139634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40b3e29a18b48cba6800c9f3a0dfee9', 'request_id': 'req-1efe0a00-8e3f-4b08-a701-7f429533e404'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.151367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] The token cache is being invalidate because project f40b3e29a18b48cba6800c9f3a0dfee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:09.151652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.152301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-1efe0a00-8e3f-4b08-a701-7f429533e404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.154466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1230/2451] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:08 2022] DELETE /identity/v3/projects/f40b3e29a18b48cba6800c9f3a0dfee9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:09.159227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559694b-97c5-4fbd-94c2-b7373e28d33c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.159517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559694b-97c5-4fbd-94c2-b7373e28d33c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.159795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559694b-97c5-4fbd-94c2-b7373e28d33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.181328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d559694b-97c5-4fbd-94c2-b7373e28d33c None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:09.182542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d559694b-97c5-4fbd-94c2-b7373e28d33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:09.242239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1222/2452] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:09.250790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719dc5a-49fc-4df3-a813-91e5abe0670a None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.251077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719dc5a-49fc-4df3-a813-91e5abe0670a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.251350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719dc5a-49fc-4df3-a813-91e5abe0670a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.271829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f719dc5a-49fc-4df3-a813-91e5abe0670a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:09.273028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f719dc5a-49fc-4df3-a813-91e5abe0670a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:09.281747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1231/2453] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:09.285986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559694b-97c5-4fbd-94c2-b7373e28d33c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.296493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559694b-97c5-4fbd-94c2-b7373e28d33c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.301225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf6a14-122b-4bb9-9df9-6ef83f4cc72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.301820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6a14-122b-4bb9-9df9-6ef83f4cc72c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.302100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6a14-122b-4bb9-9df9-6ef83f4cc72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.302367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6a14-122b-4bb9-9df9-6ef83f4cc72c admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.302998 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf6a14-122b-4bb9-9df9-6ef83f4cc72c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:09.303294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf6a14-122b-4bb9-9df9-6ef83f4cc72c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.304858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf6a14-122b-4bb9-9df9-6ef83f4cc72c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.317200 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1223/2454] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:09.321498 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719dc5a-49fc-4df3-a813-91e5abe0670a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.331228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719dc5a-49fc-4df3-a813-91e5abe0670a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.335792 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02950fd-7a32-4172-814f-64c9621da21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.336371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02950fd-7a32-4172-814f-64c9621da21a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.336644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02950fd-7a32-4172-814f-64c9621da21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.336910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02950fd-7a32-4172-814f-64c9621da21a admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.337568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02950fd-7a32-4172-814f-64c9621da21a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.339481 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02950fd-7a32-4172-814f-64c9621da21a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.366926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1232/2455] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:09.370923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf6a14-122b-4bb9-9df9-6ef83f4cc72c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.380828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf6a14-122b-4bb9-9df9-6ef83f4cc72c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.441352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea279828-23fb-4292-98db-eae347db3202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.441956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea279828-23fb-4292-98db-eae347db3202 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.442237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea279828-23fb-4292-98db-eae347db3202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.442624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea279828-23fb-4292-98db-eae347db3202 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.443258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea279828-23fb-4292-98db-eae347db3202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.445462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea279828-23fb-4292-98db-eae347db3202 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.471451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1224/2456] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:09.476645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02950fd-7a32-4172-814f-64c9621da21a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.489870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02950fd-7a32-4172-814f-64c9621da21a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.494573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456a84b-c540-4d52-ad1b-b143ffad1e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.495176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456a84b-c540-4d52-ad1b-b143ffad1e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.495456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456a84b-c540-4d52-ad1b-b143ffad1e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.495724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456a84b-c540-4d52-ad1b-b143ffad1e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.496359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456a84b-c540-4d52-ad1b-b143ffad1e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.498022 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456a84b-c540-4d52-ad1b-b143ffad1e3e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.508059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1233/2457] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:09.512334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea279828-23fb-4292-98db-eae347db3202 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.523511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea279828-23fb-4292-98db-eae347db3202 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.528233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca299095-c8a9-4ae8-8a98-bdb61f694538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.528862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca299095-c8a9-4ae8-8a98-bdb61f694538 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.529886 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca299095-c8a9-4ae8-8a98-bdb61f694538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.530151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca299095-c8a9-4ae8-8a98-bdb61f694538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.531059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca299095-c8a9-4ae8-8a98-bdb61f694538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.532631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca299095-c8a9-4ae8-8a98-bdb61f694538 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.543541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1225/2458] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:09.547852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456a84b-c540-4d52-ad1b-b143ffad1e3e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.557902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456a84b-c540-4d52-ad1b-b143ffad1e3e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.563245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95614033-5217-4237-b197-81d414f3ef42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.563766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95614033-5217-4237-b197-81d414f3ef42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.564045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95614033-5217-4237-b197-81d414f3ef42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.564304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95614033-5217-4237-b197-81d414f3ef42 admin admin] PATH_INFO: `/v3/projects/095f9e902dc944be87316ccce31b5363/users/7b67111fc8864d8f8798a36f5242c0eb/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.579781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95614033-5217-4237-b197-81d414f3ef42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095f9e902dc944be87316ccce31b5363, user_id=7b67111fc8864d8f8798a36f5242c0eb, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.582475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95614033-5217-4237-b197-81d414f3ef42 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.591894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:09.598478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1234/2459] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:09 2022] PUT /identity/v3/projects/095f9e902dc944be87316ccce31b5363/users/7b67111fc8864d8f8798a36f5242c0eb/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:09.610456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2fbc4-6ee1-4f5f-9119-15b2e0afad4a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.611014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2fbc4-6ee1-4f5f-9119-15b2e0afad4a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.611014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2fbc4-6ee1-4f5f-9119-15b2e0afad4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.624534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:09.642972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6ac2fbc4-6ee1-4f5f-9119-15b2e0afad4a None None] MFA Rules not processed for user `7b67111fc8864d8f8798a36f5242c0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:09.644178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6ac2fbc4-6ee1-4f5f-9119-15b2e0afad4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:09.706900 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1226/2460] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:09.711589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95614033-5217-4237-b197-81d414f3ef42 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.774739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95614033-5217-4237-b197-81d414f3ef42 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.779451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c8b80-c91c-4bbf-8f04-a37502aca2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.780047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c8b80-c91c-4bbf-8f04-a37502aca2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.780594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c8b80-c91c-4bbf-8f04-a37502aca2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.780869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c8b80-c91c-4bbf-8f04-a37502aca2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.781548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c8b80-c91c-4bbf-8f04-a37502aca2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.783443 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c8b80-c91c-4bbf-8f04-a37502aca2c5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.810059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1235/2461] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:09.815880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2fbc4-6ee1-4f5f-9119-15b2e0afad4a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.827179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2fbc4-6ee1-4f5f-9119-15b2e0afad4a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.881267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376156ef-7610-4870-ab1c-8d2f6b39b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.881929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376156ef-7610-4870-ab1c-8d2f6b39b830 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.882150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376156ef-7610-4870-ab1c-8d2f6b39b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.882418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376156ef-7610-4870-ab1c-8d2f6b39b830 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.883043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376156ef-7610-4870-ab1c-8d2f6b39b830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.884927 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376156ef-7610-4870-ab1c-8d2f6b39b830 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.906982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1227/2462] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:09.910837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c8b80-c91c-4bbf-8f04-a37502aca2c5 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.923868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c8b80-c91c-4bbf-8f04-a37502aca2c5 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.929207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83bc76-c484-47d1-b7e8-28ecaebf9759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.929808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83bc76-c484-47d1-b7e8-28ecaebf9759 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.930101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83bc76-c484-47d1-b7e8-28ecaebf9759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.930371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83bc76-c484-47d1-b7e8-28ecaebf9759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.931011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83bc76-c484-47d1-b7e8-28ecaebf9759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.932594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83bc76-c484-47d1-b7e8-28ecaebf9759 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.946681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1236/2463] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:09.948077 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376156ef-7610-4870-ab1c-8d2f6b39b830 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.961321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376156ef-7610-4870-ab1c-8d2f6b39b830 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.966302 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10592224-3a38-49ed-832d-a406d2c063f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.967667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10592224-3a38-49ed-832d-a406d2c063f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.967950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10592224-3a38-49ed-832d-a406d2c063f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.968227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10592224-3a38-49ed-832d-a406d2c063f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.968878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10592224-3a38-49ed-832d-a406d2c063f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.970489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10592224-3a38-49ed-832d-a406d2c063f4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.980751 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1228/2464] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:09.984910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83bc76-c484-47d1-b7e8-28ecaebf9759 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.995432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83bc76-c484-47d1-b7e8-28ecaebf9759 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:10.000668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fc807-edfb-45a1-9079-83cd54f7d341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:10.001353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fc807-edfb-45a1-9079-83cd54f7d341 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.001630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fc807-edfb-45a1-9079-83cd54f7d341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.001902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fc807-edfb-45a1-9079-83cd54f7d341 admin admin] PATH_INFO: `/v3/projects/eff654f107924838812376bc9dc1ae9d/users/1d945836c424456596ccdce038daf538/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.017324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fc807-edfb-45a1-9079-83cd54f7d341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff654f107924838812376bc9dc1ae9d, user_id=1d945836c424456596ccdce038daf538, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:10.020136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fc807-edfb-45a1-9079-83cd54f7d341 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:10.034690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1237/2465] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:09 2022] PUT /identity/v3/projects/eff654f107924838812376bc9dc1ae9d/users/1d945836c424456596ccdce038daf538/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:10.045563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7057-893d-4041-bf36-57d6ac24138c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.045853 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7057-893d-4041-bf36-57d6ac24138c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.046158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7057-893d-4041-bf36-57d6ac24138c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.077152 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-abea7057-893d-4041-bf36-57d6ac24138c None None] MFA Rules not processed for user `1d945836c424456596ccdce038daf538`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:10.078809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-abea7057-893d-4041-bf36-57d6ac24138c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:10.147911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1229/2466] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:10.152632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fc807-edfb-45a1-9079-83cd54f7d341 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:10.218190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fc807-edfb-45a1-9079-83cd54f7d341 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:10.222899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73327700-e218-43d0-8f03-6e6182d8caa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:10.223571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73327700-e218-43d0-8f03-6e6182d8caa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.232705 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73327700-e218-43d0-8f03-6e6182d8caa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.233766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73327700-e218-43d0-8f03-6e6182d8caa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.234752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73327700-e218-43d0-8f03-6e6182d8caa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:10.236383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73327700-e218-43d0-8f03-6e6182d8caa8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:10.265815 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1238/2467] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:10 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:10.269883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea7057-893d-4041-bf36-57d6ac24138c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:10.276480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:10.282486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea7057-893d-4041-bf36-57d6ac24138c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:10.338943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6d9e4-5eb3-4417-ae09-c96dfe81b7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:10.339548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6d9e4-5eb3-4417-ae09-c96dfe81b7bb admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.339827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6d9e4-5eb3-4417-ae09-c96dfe81b7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.340098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6d9e4-5eb3-4417-ae09-c96dfe81b7bb admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.340729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6d9e4-5eb3-4417-ae09-c96dfe81b7bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:10.343367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6d9e4-5eb3-4417-ae09-c96dfe81b7bb admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:10.365952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1230/2468] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:10 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:10.368472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73327700-e218-43d0-8f03-6e6182d8caa8 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:10.379560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73327700-e218-43d0-8f03-6e6182d8caa8 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:10.384237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b944-dab8-4e4a-9fbc-0f42a76a3b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:10.384827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b944-dab8-4e4a-9fbc-0f42a76a3b09 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.385138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b944-dab8-4e4a-9fbc-0f42a76a3b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.385415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b944-dab8-4e4a-9fbc-0f42a76a3b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.386052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289b944-dab8-4e4a-9fbc-0f42a76a3b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:10.387695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289b944-dab8-4e4a-9fbc-0f42a76a3b09 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:10.519573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1239/2469] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:10.523816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6d9e4-5eb3-4417-ae09-c96dfe81b7bb admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:10.534120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6d9e4-5eb3-4417-ae09-c96dfe81b7bb admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:10.539910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb14ab-a59e-49b7-93f5-07f6736a0a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:10.540554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb14ab-a59e-49b7-93f5-07f6736a0a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.540830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb14ab-a59e-49b7-93f5-07f6736a0a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.541134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb14ab-a59e-49b7-93f5-07f6736a0a82 admin admin] PATH_INFO: `/v3/projects/7a4c7e3e066f4469afa4d1b839545329/users/5f24a7c5241743f7addc4f9fdd3752dc/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.555954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb14ab-a59e-49b7-93f5-07f6736a0a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a4c7e3e066f4469afa4d1b839545329, user_id=5f24a7c5241743f7addc4f9fdd3752dc, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:10.558694 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb14ab-a59e-49b7-93f5-07f6736a0a82 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:10.572138 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1231/2470] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:10 2022] PUT /identity/v3/projects/7a4c7e3e066f4469afa4d1b839545329/users/5f24a7c5241743f7addc4f9fdd3752dc/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:10.576736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b944-dab8-4e4a-9fbc-0f42a76a3b09 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:10.586903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b944-dab8-4e4a-9fbc-0f42a76a3b09 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:10.605588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:10.643768 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f3cdf-b8be-49f6-8d62-2834dd1b6b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:10.644287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f3cdf-b8be-49f6-8d62-2834dd1b6b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.644564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f3cdf-b8be-49f6-8d62-2834dd1b6b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.644831 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f3cdf-b8be-49f6-8d62-2834dd1b6b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.645497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f3cdf-b8be-49f6-8d62-2834dd1b6b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:10.647075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f3cdf-b8be-49f6-8d62-2834dd1b6b0f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:10.659252 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1240/2471] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:10.663417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb14ab-a59e-49b7-93f5-07f6736a0a82 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:10.674551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb14ab-a59e-49b7-93f5-07f6736a0a82 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:10.679331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd157a-dadd-46e8-8671-10f7c5690c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:10.680042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd157a-dadd-46e8-8671-10f7c5690c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.680314 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd157a-dadd-46e8-8671-10f7c5690c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.680592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd157a-dadd-46e8-8671-10f7c5690c9c admin admin] PATH_INFO: `/v3/projects/7a4c7e3e066f4469afa4d1b839545329/users/5f24a7c5241743f7addc4f9fdd3752dc/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.682522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd157a-dadd-46e8-8671-10f7c5690c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a4c7e3e066f4469afa4d1b839545329, user_id=5f24a7c5241743f7addc4f9fdd3752dc, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:10.685162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd157a-dadd-46e8-8671-10f7c5690c9c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:10.696062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1232/2472] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:10 2022] PUT /identity/v3/projects/7a4c7e3e066f4469afa4d1b839545329/users/5f24a7c5241743f7addc4f9fdd3752dc/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:10.707372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac533a-8dfd-4166-ae50-7dd2b7cf7024 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.707750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac533a-8dfd-4166-ae50-7dd2b7cf7024 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.708088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac533a-8dfd-4166-ae50-7dd2b7cf7024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.738453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-89ac533a-8dfd-4166-ae50-7dd2b7cf7024 None None] MFA Rules not processed for user `5f24a7c5241743f7addc4f9fdd3752dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:10.739658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-89ac533a-8dfd-4166-ae50-7dd2b7cf7024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:10.824456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1241/2473] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:10.841002 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd157a-dadd-46e8-8671-10f7c5690c9c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:10.843394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1cfd157a-dadd-46e8-8671-10f7c5690c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:10.900353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd157a-dadd-46e8-8671-10f7c5690c9c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:10.904059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366d41-a8a3-4a77-9faf-562116b0ae60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:10.905486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366d41-a8a3-4a77-9faf-562116b0ae60 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.905709 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366d41-a8a3-4a77-9faf-562116b0ae60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.905974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366d41-a8a3-4a77-9faf-562116b0ae60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.915001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59366d41-a8a3-4a77-9faf-562116b0ae60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:10.916428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59366d41-a8a3-4a77-9faf-562116b0ae60 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:10.926212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1233/2474] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:11.443182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac533a-8dfd-4166-ae50-7dd2b7cf7024 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:11.444594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-89ac533a-8dfd-4166-ae50-7dd2b7cf7024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:11.456508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac533a-8dfd-4166-ae50-7dd2b7cf7024 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:11.459983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c3146-789b-40fd-bc18-c454ef42a7a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:11.460600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c3146-789b-40fd-bc18-c454ef42a7a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:11.460873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c3146-789b-40fd-bc18-c454ef42a7a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:11.461169 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c3146-789b-40fd-bc18-c454ef42a7a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:11.517397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c3146-789b-40fd-bc18-c454ef42a7a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:11.518812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c3146-789b-40fd-bc18-c454ef42a7a2 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:11.527484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1242/2475] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:16.218055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366d41-a8a3-4a77-9faf-562116b0ae60 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:16.219494 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-59366d41-a8a3-4a77-9faf-562116b0ae60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:16.288293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366d41-a8a3-4a77-9faf-562116b0ae60 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:16.292783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:16.293436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:16.293715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:16.293986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] PATH_INFO: `/v3/users/eaed78fa27894d4f9ccc5375d8bc17a6` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:16.295824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaed78fa27894d4f9ccc5375d8bc17a6)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:16.297790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:16.299300 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:16.347454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaed78fa27894d4f9ccc5375d8bc17a6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:16.356082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaed78fa27894d4f9ccc5375d8bc17a6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:16.356699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaed78fa27894d4f9ccc5375d8bc17a6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:16.358006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaed78fa27894d4f9ccc5375d8bc17a6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:16.369929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaed78fa27894d4f9ccc5375d8bc17a6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:16.382740 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1234/2476] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:16 2022] DELETE /identity/v3/users/eaed78fa27894d4f9ccc5375d8bc17a6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:16.387294 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17268252-ae3c-4bd6-9082-1d69c94795d1 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:16.387756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17268252-ae3c-4bd6-9082-1d69c94795d1 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:16.387889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17268252-ae3c-4bd6-9082-1d69c94795d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:16.407787 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-17268252-ae3c-4bd6-9082-1d69c94795d1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:16.408955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-17268252-ae3c-4bd6-9082-1d69c94795d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:16.447507 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:16.470238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1243/2477] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:16.480183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:16.481601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:16.561323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb9b35-4b62-4244-81a1-7667717b09de admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:16.561323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfbae5-86fe-4749-8a8b-5e15868653b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:16.561323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfbae5-86fe-4749-8a8b-5e15868653b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:16.561323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfbae5-86fe-4749-8a8b-5e15868653b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:16.561654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfbae5-86fe-4749-8a8b-5e15868653b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:16.570660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfbae5-86fe-4749-8a8b-5e15868653b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:16.572080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfbae5-86fe-4749-8a8b-5e15868653b9 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:16.583873 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1235/2478] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:16.633947 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17268252-ae3c-4bd6-9082-1d69c94795d1 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:16.635456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-17268252-ae3c-4bd6-9082-1d69c94795d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:16.697836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17268252-ae3c-4bd6-9082-1d69c94795d1 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:16.702304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6353939-0409-4ee5-9d3b-ba95ead8b588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:16.702910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6353939-0409-4ee5-9d3b-ba95ead8b588 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:16.703190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6353939-0409-4ee5-9d3b-ba95ead8b588 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:16.703458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6353939-0409-4ee5-9d3b-ba95ead8b588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:16.705242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b6353939-0409-4ee5-9d3b-ba95ead8b588 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:16.771455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6353939-0409-4ee5-9d3b-ba95ead8b588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:16.773019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6353939-0409-4ee5-9d3b-ba95ead8b588 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:16.789228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1244/2479] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:16.938851 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfbae5-86fe-4749-8a8b-5e15868653b9 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:16.941121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-16cfbae5-86fe-4749-8a8b-5e15868653b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:16.960298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfbae5-86fe-4749-8a8b-5e15868653b9 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:16.967184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:16.967968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:16.968229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:16.968487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] PATH_INFO: `/v3/projects/12e6304ad08b4ebcaa9218f84fc191e0` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:16.970664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] RBAC: Authorizing `identity:delete_project(project_id=12e6304ad08b4ebcaa9218f84fc191e0)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:16.972280 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:17.025688 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12e6304ad08b4ebcaa9218f84fc191e0', 'request_id': 'req-98c77cfe-155d-4278-9c51-f0a3be64e07e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.032863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12e6304ad08b4ebcaa9218f84fc191e0', 'request_id': 'req-98c77cfe-155d-4278-9c51-f0a3be64e07e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.040003 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12e6304ad08b4ebcaa9218f84fc191e0', 'request_id': 'req-98c77cfe-155d-4278-9c51-f0a3be64e07e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.051290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] The token cache is being invalidate because project 12e6304ad08b4ebcaa9218f84fc191e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:17.051579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.052245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.053795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1236/2480] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:16 2022] DELETE /identity/v3/projects/12e6304ad08b4ebcaa9218f84fc191e0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:17.058280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6353939-0409-4ee5-9d3b-ba95ead8b588 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:17.070338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6353939-0409-4ee5-9d3b-ba95ead8b588 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:17.133174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:17.133799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:17.134106 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:17.134410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] PATH_INFO: `/v3/users/0c0c4a6733c048cda59a80e351995463` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:17.135582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0c4a6733c048cda59a80e351995463)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:17.137860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:17.180465 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:17.188134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0c4a6733c048cda59a80e351995463'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.196700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0c4a6733c048cda59a80e351995463'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.205854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0c4a6733c048cda59a80e351995463'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.206621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0c4a6733c048cda59a80e351995463'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.225728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0bf65e58-54b0-4528-b624-8b99a8f45164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0c4a6733c048cda59a80e351995463'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.231920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1245/2481] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:17 2022] DELETE /identity/v3/users/0c0c4a6733c048cda59a80e351995463 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:17.531883 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:17.533381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:17.542680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:17.551848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77cfe-155d-4278-9c51-f0a3be64e07e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:17.615019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:17.615881 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:17.616148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:17.616555 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] PATH_INFO: `/v3/projects/f1a9dfae25d545cbb0ffc14973ea2f8b` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:17.617789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a9dfae25d545cbb0ffc14973ea2f8b)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:17.619780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:17.671698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a9dfae25d545cbb0ffc14973ea2f8b', 'request_id': 'req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.678214 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a9dfae25d545cbb0ffc14973ea2f8b', 'request_id': 'req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.686698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a9dfae25d545cbb0ffc14973ea2f8b', 'request_id': 'req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.696835 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] The token cache is being invalidate because project f1a9dfae25d545cbb0ffc14973ea2f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:17.697155 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.697773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4c57ec81-4084-4b81-b6a7-94dc55ac45cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:17.699244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1237/2482] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:17 2022] DELETE /identity/v3/projects/f1a9dfae25d545cbb0ffc14973ea2f8b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:17.704178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6f42-8fe9-4a2f-b2f9-604b12a2857b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:17.704695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6f42-8fe9-4a2f-b2f9-604b12a2857b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:17.704804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6f42-8fe9-4a2f-b2f9-604b12a2857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:17.724087 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3b4f6f42-8fe9-4a2f-b2f9-604b12a2857b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:17.725344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3b4f6f42-8fe9-4a2f-b2f9-604b12a2857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:17.783970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1246/2483] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:17.793892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ca7cf-3fe7-4f47-8794-92436a15ff54 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:17.794319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ca7cf-3fe7-4f47-8794-92436a15ff54 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:17.794689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ca7cf-3fe7-4f47-8794-92436a15ff54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:17.819369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b75ca7cf-3fe7-4f47-8794-92436a15ff54 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:17.820573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b75ca7cf-3fe7-4f47-8794-92436a15ff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:17.828846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1238/2484] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:17.833055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f6f42-8fe9-4a2f-b2f9-604b12a2857b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:17.842993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f6f42-8fe9-4a2f-b2f9-604b12a2857b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:17.847437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658c28e-a5ba-4467-8456-71b606620751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:17.848024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658c28e-a5ba-4467-8456-71b606620751 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:17.848296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658c28e-a5ba-4467-8456-71b606620751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:17.848561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658c28e-a5ba-4467-8456-71b606620751 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:17.849237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658c28e-a5ba-4467-8456-71b606620751 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:17.849536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658c28e-a5ba-4467-8456-71b606620751 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:17.851102 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658c28e-a5ba-4467-8456-71b606620751 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:17.863561 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1247/2485] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:17.869179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ca7cf-3fe7-4f47-8794-92436a15ff54 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:17.879285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ca7cf-3fe7-4f47-8794-92436a15ff54 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:17.883866 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330cf07-a6f3-435d-881a-626151aef823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:17.884460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330cf07-a6f3-435d-881a-626151aef823 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:17.884736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330cf07-a6f3-435d-881a-626151aef823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:17.885006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330cf07-a6f3-435d-881a-626151aef823 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:17.885676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330cf07-a6f3-435d-881a-626151aef823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:17.887568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330cf07-a6f3-435d-881a-626151aef823 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:17.916958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1239/2486] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:17 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:17.921801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658c28e-a5ba-4467-8456-71b606620751 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:17.932237 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658c28e-a5ba-4467-8456-71b606620751 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:17.989986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b40b5-0782-492b-8440-7302c6716dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:17.990587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b40b5-0782-492b-8440-7302c6716dac admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:17.990861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b40b5-0782-492b-8440-7302c6716dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:17.991126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b40b5-0782-492b-8440-7302c6716dac admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:17.991754 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b40b5-0782-492b-8440-7302c6716dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:17.993703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b40b5-0782-492b-8440-7302c6716dac admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.014266 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1248/2487] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:17 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:18.018533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330cf07-a6f3-435d-881a-626151aef823 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.030345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330cf07-a6f3-435d-881a-626151aef823 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.034997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25959818-0b00-4c8d-8480-6e07f7b2b3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.035585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25959818-0b00-4c8d-8480-6e07f7b2b3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.035864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25959818-0b00-4c8d-8480-6e07f7b2b3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.036137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25959818-0b00-4c8d-8480-6e07f7b2b3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.036771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25959818-0b00-4c8d-8480-6e07f7b2b3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.038380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25959818-0b00-4c8d-8480-6e07f7b2b3f6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.049426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1240/2488] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:18.053271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b40b5-0782-492b-8440-7302c6716dac admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.063763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b40b5-0782-492b-8440-7302c6716dac admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.068349 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f61e9-4381-4413-a545-4097e1b5e4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.068935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f61e9-4381-4413-a545-4097e1b5e4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.069248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f61e9-4381-4413-a545-4097e1b5e4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.069515 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f61e9-4381-4413-a545-4097e1b5e4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.070145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f61e9-4381-4413-a545-4097e1b5e4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.071727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f61e9-4381-4413-a545-4097e1b5e4f1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.082455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1249/2489] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:18.086481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25959818-0b00-4c8d-8480-6e07f7b2b3f6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.096997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25959818-0b00-4c8d-8480-6e07f7b2b3f6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.101624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b124f-77bc-417c-94fb-4c580bcf17a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.102344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b124f-77bc-417c-94fb-4c580bcf17a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.102630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b124f-77bc-417c-94fb-4c580bcf17a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.102902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b124f-77bc-417c-94fb-4c580bcf17a1 admin admin] PATH_INFO: `/v3/projects/dd28b75bd4644ebf9012e68aae8c3fb9/users/a41a18aa9bc543e987e17082f9c990ed/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.117935 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b124f-77bc-417c-94fb-4c580bcf17a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd28b75bd4644ebf9012e68aae8c3fb9, user_id=a41a18aa9bc543e987e17082f9c990ed, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.120555 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b124f-77bc-417c-94fb-4c580bcf17a1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.133130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1241/2490] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:18 2022] PUT /identity/v3/projects/dd28b75bd4644ebf9012e68aae8c3fb9/users/a41a18aa9bc543e987e17082f9c990ed/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:18.144340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc145b-7146-4307-917d-4f186d781375 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.144624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc145b-7146-4307-917d-4f186d781375 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.144895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc145b-7146-4307-917d-4f186d781375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.173511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-40dc145b-7146-4307-917d-4f186d781375 None None] MFA Rules not processed for user `a41a18aa9bc543e987e17082f9c990ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:18.174647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-40dc145b-7146-4307-917d-4f186d781375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:18.183546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:18.247586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1250/2491] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:18.252739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b124f-77bc-417c-94fb-4c580bcf17a1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.286246 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:18.318182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b124f-77bc-417c-94fb-4c580bcf17a1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.323034 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a239e-ca11-4136-a512-1894cd93baa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.323676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a239e-ca11-4136-a512-1894cd93baa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.323987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a239e-ca11-4136-a512-1894cd93baa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.324256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a239e-ca11-4136-a512-1894cd93baa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.324885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a239e-ca11-4136-a512-1894cd93baa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.327369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a239e-ca11-4136-a512-1894cd93baa2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.354360 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1242/2492] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:18 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:18.358322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc145b-7146-4307-917d-4f186d781375 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.369021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc145b-7146-4307-917d-4f186d781375 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.424720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8d1c5-421d-4631-ac2c-738fa6626835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.425273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8d1c5-421d-4631-ac2c-738fa6626835 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.425549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8d1c5-421d-4631-ac2c-738fa6626835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.425812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8d1c5-421d-4631-ac2c-738fa6626835 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.426423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8d1c5-421d-4631-ac2c-738fa6626835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.428344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8d1c5-421d-4631-ac2c-738fa6626835 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.449596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1251/2493] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:18 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:18.454773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a239e-ca11-4136-a512-1894cd93baa2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.465326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a239e-ca11-4136-a512-1894cd93baa2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.470769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e4d8b-1f35-4022-8ffa-a4b9436facbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.471369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e4d8b-1f35-4022-8ffa-a4b9436facbd admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.471648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e4d8b-1f35-4022-8ffa-a4b9436facbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.472411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e4d8b-1f35-4022-8ffa-a4b9436facbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.473101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e4d8b-1f35-4022-8ffa-a4b9436facbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.474747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e4d8b-1f35-4022-8ffa-a4b9436facbd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.489934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1243/2494] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:18.492894 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8d1c5-421d-4631-ac2c-738fa6626835 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.506114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8d1c5-421d-4631-ac2c-738fa6626835 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.511933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f07ac-d97d-41d3-be56-f9e5dcff89d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.512593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f07ac-d97d-41d3-be56-f9e5dcff89d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.512948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f07ac-d97d-41d3-be56-f9e5dcff89d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.513749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f07ac-d97d-41d3-be56-f9e5dcff89d2 admin admin] PATH_INFO: `/v3/projects/72377d04b72c4c0aaf2bdff9321039a8/users/c08fb7c89fb64799ac13e421f83f655d/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.531611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f07ac-d97d-41d3-be56-f9e5dcff89d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72377d04b72c4c0aaf2bdff9321039a8, user_id=c08fb7c89fb64799ac13e421f83f655d, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.534743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f07ac-d97d-41d3-be56-f9e5dcff89d2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.554111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1252/2495] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:18 2022] PUT /identity/v3/projects/72377d04b72c4c0aaf2bdff9321039a8/users/c08fb7c89fb64799ac13e421f83f655d/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:18.558149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e4d8b-1f35-4022-8ffa-a4b9436facbd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.570597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e4d8b-1f35-4022-8ffa-a4b9436facbd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.641112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb784a3-9752-47d2-99fa-1e07f05b7d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.641735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb784a3-9752-47d2-99fa-1e07f05b7d79 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.642086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb784a3-9752-47d2-99fa-1e07f05b7d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.642450 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb784a3-9752-47d2-99fa-1e07f05b7d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.643164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb784a3-9752-47d2-99fa-1e07f05b7d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.644890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb784a3-9752-47d2-99fa-1e07f05b7d79 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.659702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1244/2496] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:08:18.660735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f07ac-d97d-41d3-be56-f9e5dcff89d2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.671604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f07ac-d97d-41d3-be56-f9e5dcff89d2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.676249 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234f53-5df3-4f68-8e9e-ab41c085b9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.676822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234f53-5df3-4f68-8e9e-ab41c085b9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.677122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234f53-5df3-4f68-8e9e-ab41c085b9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.677395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234f53-5df3-4f68-8e9e-ab41c085b9d1 admin admin] PATH_INFO: `/v3/projects/72377d04b72c4c0aaf2bdff9321039a8/users/c08fb7c89fb64799ac13e421f83f655d/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.679411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe234f53-5df3-4f68-8e9e-ab41c085b9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72377d04b72c4c0aaf2bdff9321039a8, user_id=c08fb7c89fb64799ac13e421f83f655d, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.682153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe234f53-5df3-4f68-8e9e-ab41c085b9d1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.693375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1253/2497] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:18 2022] PUT /identity/v3/projects/72377d04b72c4c0aaf2bdff9321039a8/users/c08fb7c89fb64799ac13e421f83f655d/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:18.706197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aad9d3-320a-4973-a318-e42f57b44703 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.706489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aad9d3-320a-4973-a318-e42f57b44703 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.706781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aad9d3-320a-4973-a318-e42f57b44703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.737051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-02aad9d3-320a-4973-a318-e42f57b44703 None None] MFA Rules not processed for user `c08fb7c89fb64799ac13e421f83f655d`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:18.738292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-02aad9d3-320a-4973-a318-e42f57b44703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:18.816614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1245/2498] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:18.829001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234f53-5df3-4f68-8e9e-ab41c085b9d1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.830887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fe234f53-5df3-4f68-8e9e-ab41c085b9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:18.894098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234f53-5df3-4f68-8e9e-ab41c085b9d1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.897945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785c4b8-6e83-45ac-b3eb-01cb6e70d4a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.898649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785c4b8-6e83-45ac-b3eb-01cb6e70d4a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.898992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785c4b8-6e83-45ac-b3eb-01cb6e70d4a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.899329 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785c4b8-6e83-45ac-b3eb-01cb6e70d4a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.909283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785c4b8-6e83-45ac-b3eb-01cb6e70d4a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.910827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785c4b8-6e83-45ac-b3eb-01cb6e70d4a3 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.921810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1254/2499] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:18.964720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aad9d3-320a-4973-a318-e42f57b44703 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.966374 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-02aad9d3-320a-4973-a318-e42f57b44703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:19.002663 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785c4b8-6e83-45ac-b3eb-01cb6e70d4a3 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:19.004645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0785c4b8-6e83-45ac-b3eb-01cb6e70d4a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:19.007055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:19.018247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785c4b8-6e83-45ac-b3eb-01cb6e70d4a3 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:19.022438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e975c-dd30-4817-b2ce-28f956473763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:19.023043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e975c-dd30-4817-b2ce-28f956473763 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:19.023317 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e975c-dd30-4817-b2ce-28f956473763 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:19.023587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e975c-dd30-4817-b2ce-28f956473763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:19.028186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:19.047079 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aad9d3-320a-4973-a318-e42f57b44703 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:19.052491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:19.053143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:19.053423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:19.053692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] PATH_INFO: `/v3/users/7b67111fc8864d8f8798a36f5242c0eb` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:19.054831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b67111fc8864d8f8798a36f5242c0eb)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:19.056721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:19.094288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e975c-dd30-4817-b2ce-28f956473763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:19.096039 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e975c-dd30-4817-b2ce-28f956473763 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:19.107790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1255/2500] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:08:19.116148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b67111fc8864d8f8798a36f5242c0eb'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.126034 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b67111fc8864d8f8798a36f5242c0eb'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.126725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b67111fc8864d8f8798a36f5242c0eb'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.128212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b67111fc8864d8f8798a36f5242c0eb'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.144953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b67111fc8864d8f8798a36f5242c0eb'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.159423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1246/2501] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:18 2022] DELETE /identity/v3/users/7b67111fc8864d8f8798a36f5242c0eb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:19.169727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1247d3-ef2a-487a-81b7-763ae75c045e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:19.170021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1247d3-ef2a-487a-81b7-763ae75c045e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:19.170323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1247d3-ef2a-487a-81b7-763ae75c045e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:19.192820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-bd1247d3-ef2a-487a-81b7-763ae75c045e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:19.194098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bd1247d3-ef2a-487a-81b7-763ae75c045e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:19.274260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1256/2502] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:19.284428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:19.285902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:19.344051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840e4c9-934f-4b45-ab59-04c25dce4980 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:19.347722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32de1-8e17-4f10-bccb-f60f7ead97fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:19.348320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32de1-8e17-4f10-bccb-f60f7ead97fb service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:19.348601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32de1-8e17-4f10-bccb-f60f7ead97fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:19.348874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32de1-8e17-4f10-bccb-f60f7ead97fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:19.358014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c32de1-8e17-4f10-bccb-f60f7ead97fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:19.359441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c32de1-8e17-4f10-bccb-f60f7ead97fb service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:19.368824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1247/2503] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:19.637037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1247d3-ef2a-487a-81b7-763ae75c045e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:19.639141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-bd1247d3-ef2a-487a-81b7-763ae75c045e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:19.652866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1247d3-ef2a-487a-81b7-763ae75c045e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:19.658250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:19.659412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:19.659883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:19.660161 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] PATH_INFO: `/v3/projects/095f9e902dc944be87316ccce31b5363` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:19.661343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=095f9e902dc944be87316ccce31b5363)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:19.663264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:19.711451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095f9e902dc944be87316ccce31b5363', 'request_id': 'req-b5076300-e24b-4a5d-861e-1c3099deaf01'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.717693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095f9e902dc944be87316ccce31b5363', 'request_id': 'req-b5076300-e24b-4a5d-861e-1c3099deaf01'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.724774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095f9e902dc944be87316ccce31b5363', 'request_id': 'req-b5076300-e24b-4a5d-861e-1c3099deaf01'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.735478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] The token cache is being invalidate because project 095f9e902dc944be87316ccce31b5363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:19.735765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.736405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.737975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1257/2504] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:19 2022] DELETE /identity/v3/projects/095f9e902dc944be87316ccce31b5363 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:19.741944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32de1-8e17-4f10-bccb-f60f7ead97fb service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:19.752379 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32de1-8e17-4f10-bccb-f60f7ead97fb service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:19.797390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:19.811113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:19.811731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:19.812011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:19.812285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] PATH_INFO: `/v3/users/1d945836c424456596ccdce038daf538` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:19.813444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d945836c424456596ccdce038daf538)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:19.815343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:19.861486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d945836c424456596ccdce038daf538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.868512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d945836c424456596ccdce038daf538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.869165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d945836c424456596ccdce038daf538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.869769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d945836c424456596ccdce038daf538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.882735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d945836c424456596ccdce038daf538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:19.896589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1248/2505] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:19 2022] DELETE /identity/v3/users/1d945836c424456596ccdce038daf538 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:20.384786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:20.386824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:20.410827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5076300-e24b-4a5d-861e-1c3099deaf01 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:20.461229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:20.515824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:20.516867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:20.517280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:20.517653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] PATH_INFO: `/v3/projects/eff654f107924838812376bc9dc1ae9d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:20.519281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff654f107924838812376bc9dc1ae9d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:20.521372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:20.593581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff654f107924838812376bc9dc1ae9d', 'request_id': 'req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.601716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff654f107924838812376bc9dc1ae9d', 'request_id': 'req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.614829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff654f107924838812376bc9dc1ae9d', 'request_id': 'req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.657683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] The token cache is being invalidate because project eff654f107924838812376bc9dc1ae9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:20.658243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.660279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d188fe2d-a6c2-4114-ac47-e88e2e29b69f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.663760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1258/2506] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:20 2022] DELETE /identity/v3/projects/eff654f107924838812376bc9dc1ae9d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:20.667977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:20.670028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:20.689537 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c12c5b-a568-4cb2-8b96-c0341ec80965 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:20.751730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:20.752385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:20.752669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:20.752887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] PATH_INFO: `/v3/users/5f24a7c5241743f7addc4f9fdd3752dc` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:20.754073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f24a7c5241743f7addc4f9fdd3752dc)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:20.756620 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:20.861526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f24a7c5241743f7addc4f9fdd3752dc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.873577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f24a7c5241743f7addc4f9fdd3752dc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.874422 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f24a7c5241743f7addc4f9fdd3752dc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.875174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f24a7c5241743f7addc4f9fdd3752dc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.896199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:20.898201 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f24a7c5241743f7addc4f9fdd3752dc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:20.914153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1249/2507] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:20 2022] DELETE /identity/v3/users/5f24a7c5241743f7addc4f9fdd3752dc => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:20.935715 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03302466-5ffe-4f0c-8d2b-95f5bbf7fb67 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:20.935986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03302466-5ffe-4f0c-8d2b-95f5bbf7fb67 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:20.936351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03302466-5ffe-4f0c-8d2b-95f5bbf7fb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:20.957014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-03302466-5ffe-4f0c-8d2b-95f5bbf7fb67 None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:20.958269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-03302466-5ffe-4f0c-8d2b-95f5bbf7fb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:21.018721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1259/2508] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:08:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:21.072982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:21.074587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:21.162348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0a279-46dc-4cda-9ae0-5d5e75c5bc1c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:21.167510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3b6e2-699e-499b-ba0d-5f62d498e2d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:21.168118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3b6e2-699e-499b-ba0d-5f62d498e2d0 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.168428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3b6e2-699e-499b-ba0d-5f62d498e2d0 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.168702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3b6e2-699e-499b-ba0d-5f62d498e2d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.177937 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3b6e2-699e-499b-ba0d-5f62d498e2d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:21.179513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3b6e2-699e-499b-ba0d-5f62d498e2d0 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:21.189444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1250/2509] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:08:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:08:21.210089 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03302466-5ffe-4f0c-8d2b-95f5bbf7fb67 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:21.211577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-03302466-5ffe-4f0c-8d2b-95f5bbf7fb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:21.318170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03302466-5ffe-4f0c-8d2b-95f5bbf7fb67 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:21.323444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:21.324166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.324668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.325006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] PATH_INFO: `/v3/projects/7a4c7e3e066f4469afa4d1b839545329` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.327116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a4c7e3e066f4469afa4d1b839545329)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:21.329257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:21.339094 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:21.427130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a4c7e3e066f4469afa4d1b839545329', 'request_id': 'req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:21.434518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a4c7e3e066f4469afa4d1b839545329', 'request_id': 'req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:21.450316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a4c7e3e066f4469afa4d1b839545329', 'request_id': 'req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:21.471285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] The token cache is being invalidate because project 7a4c7e3e066f4469afa4d1b839545329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:21.471640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:21.473485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2a27e9f7-7548-4b3c-a773-ee9a5ec1f756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:21.480083 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1260/2510] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:21 2022] DELETE /identity/v3/projects/7a4c7e3e066f4469afa4d1b839545329 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:21.481775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c0219-dbcc-4296-8376-e31838cf8ed5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.486477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c0219-dbcc-4296-8376-e31838cf8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.486787 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c0219-dbcc-4296-8376-e31838cf8ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.516702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e48c0219-dbcc-4296-8376-e31838cf8ed5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:21.517265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e48c0219-dbcc-4296-8376-e31838cf8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:21.606605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1251/2511] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:21.615668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fe3a3-ac7d-4131-a7bc-1e3328ead4f6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.616024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fe3a3-ac7d-4131-a7bc-1e3328ead4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.616365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fe3a3-ac7d-4131-a7bc-1e3328ead4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.640565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-929fe3a3-ac7d-4131-a7bc-1e3328ead4f6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:21.641978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-929fe3a3-ac7d-4131-a7bc-1e3328ead4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:21.650456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1261/2512] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:21.655330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c0219-dbcc-4296-8376-e31838cf8ed5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:21.666103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c0219-dbcc-4296-8376-e31838cf8ed5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:21.671298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ad6ec-fa21-456f-afd5-0c9ed27adf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:21.671979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ad6ec-fa21-456f-afd5-0c9ed27adf15 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.672264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ad6ec-fa21-456f-afd5-0c9ed27adf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.672535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ad6ec-fa21-456f-afd5-0c9ed27adf15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.673231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ad6ec-fa21-456f-afd5-0c9ed27adf15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:21.673568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ad6ec-fa21-456f-afd5-0c9ed27adf15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:21.675146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ad6ec-fa21-456f-afd5-0c9ed27adf15 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:21.689276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1252/2513] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:21.693265 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929fe3a3-ac7d-4131-a7bc-1e3328ead4f6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:21.703234 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929fe3a3-ac7d-4131-a7bc-1e3328ead4f6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:21.707878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefe087-f73d-4993-ab73-b31c1a2a2c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:21.708477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefe087-f73d-4993-ab73-b31c1a2a2c7e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.708779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefe087-f73d-4993-ab73-b31c1a2a2c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.709046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefe087-f73d-4993-ab73-b31c1a2a2c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.709753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefe087-f73d-4993-ab73-b31c1a2a2c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:21.711817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefe087-f73d-4993-ab73-b31c1a2a2c7e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:21.742680 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1262/2514] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:21 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:21.746824 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ad6ec-fa21-456f-afd5-0c9ed27adf15 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:21.759015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ad6ec-fa21-456f-afd5-0c9ed27adf15 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:21.833781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9831b5-b767-4ca8-abd4-53ad37e06020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:21.834794 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9831b5-b767-4ca8-abd4-53ad37e06020 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.835223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9831b5-b767-4ca8-abd4-53ad37e06020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.835838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9831b5-b767-4ca8-abd4-53ad37e06020 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.836470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9831b5-b767-4ca8-abd4-53ad37e06020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:21.838383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9831b5-b767-4ca8-abd4-53ad37e06020 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:21.849928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:21.864625 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1253/2515] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:21 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:21.869047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefe087-f73d-4993-ab73-b31c1a2a2c7e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:21.881185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefe087-f73d-4993-ab73-b31c1a2a2c7e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:21.885928 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3096d5-4f61-461e-b079-58fc1a3de074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:21.886526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3096d5-4f61-461e-b079-58fc1a3de074 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.886795 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3096d5-4f61-461e-b079-58fc1a3de074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.887059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3096d5-4f61-461e-b079-58fc1a3de074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.887697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3096d5-4f61-461e-b079-58fc1a3de074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:21.890014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3096d5-4f61-461e-b079-58fc1a3de074 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:21.904434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1263/2516] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:21.905877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9831b5-b767-4ca8-abd4-53ad37e06020 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:21.920949 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9831b5-b767-4ca8-abd4-53ad37e06020 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:21.926208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316ec33-f346-4695-82e0-a84a2675b6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:21.926889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316ec33-f346-4695-82e0-a84a2675b6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.927231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316ec33-f346-4695-82e0-a84a2675b6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.927568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316ec33-f346-4695-82e0-a84a2675b6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.928382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316ec33-f346-4695-82e0-a84a2675b6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:21.930059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316ec33-f346-4695-82e0-a84a2675b6a3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:21.947498 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1254/2517] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:21.950204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3096d5-4f61-461e-b079-58fc1a3de074 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:21.965556 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3096d5-4f61-461e-b079-58fc1a3de074 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:21.993862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffd58d-fe6d-4275-be98-74bc8565c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:21.994791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffd58d-fe6d-4275-be98-74bc8565c384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.995493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffd58d-fe6d-4275-be98-74bc8565c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.995834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffd58d-fe6d-4275-be98-74bc8565c384 admin admin] PATH_INFO: `/v3/projects/aff191159f404c67b2721d2d52208964/users/2b46cac717084f759335e6ba85863f57/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.017060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffd58d-fe6d-4275-be98-74bc8565c384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff191159f404c67b2721d2d52208964, user_id=2b46cac717084f759335e6ba85863f57, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:22.020021 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffd58d-fe6d-4275-be98-74bc8565c384 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:22.038966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1264/2518] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:21 2022] PUT /identity/v3/projects/aff191159f404c67b2721d2d52208964/users/2b46cac717084f759335e6ba85863f57/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:22.051043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619e393-4aec-46c0-86cd-6cbdefff4a45 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:22.051421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619e393-4aec-46c0-86cd-6cbdefff4a45 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:22.051766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619e393-4aec-46c0-86cd-6cbdefff4a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.103062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3619e393-4aec-46c0-86cd-6cbdefff4a45 None None] MFA Rules not processed for user `2b46cac717084f759335e6ba85863f57`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:22.104383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3619e393-4aec-46c0-86cd-6cbdefff4a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:22.217693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1255/2519] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:22.219397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffd58d-fe6d-4275-be98-74bc8565c384 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:22.369487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffd58d-fe6d-4275-be98-74bc8565c384 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:22.376644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7881705-d8c3-4284-bebc-64e29e45ebdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:22.377303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881705-d8c3-4284-bebc-64e29e45ebdd admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:22.377583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881705-d8c3-4284-bebc-64e29e45ebdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:22.378245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881705-d8c3-4284-bebc-64e29e45ebdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.379188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7881705-d8c3-4284-bebc-64e29e45ebdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:22.380969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7881705-d8c3-4284-bebc-64e29e45ebdd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:22.425987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1265/2520] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:22 2022] POST /identity/v3/projects => generated 393 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:22.430073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619e393-4aec-46c0-86cd-6cbdefff4a45 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:22.446332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619e393-4aec-46c0-86cd-6cbdefff4a45 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:22.543426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0e81e-88e0-4ccc-aa04-9fcfa3fab911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:22.544038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0e81e-88e0-4ccc-aa04-9fcfa3fab911 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:22.544621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0e81e-88e0-4ccc-aa04-9fcfa3fab911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:22.544621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0e81e-88e0-4ccc-aa04-9fcfa3fab911 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.545261 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0e81e-88e0-4ccc-aa04-9fcfa3fab911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:22.547212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0e81e-88e0-4ccc-aa04-9fcfa3fab911 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:22.588513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1256/2521] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:22 2022] POST /identity/v3/users => generated 303 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:22.593231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7881705-d8c3-4284-bebc-64e29e45ebdd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:22.604052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7881705-d8c3-4284-bebc-64e29e45ebdd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:22.611319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb18c9-e939-41db-8b7b-4a64a51ed6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:22.612097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb18c9-e939-41db-8b7b-4a64a51ed6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:22.612097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb18c9-e939-41db-8b7b-4a64a51ed6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:22.612280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb18c9-e939-41db-8b7b-4a64a51ed6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.613107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb18c9-e939-41db-8b7b-4a64a51ed6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:22.614739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb18c9-e939-41db-8b7b-4a64a51ed6fc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:22.627804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1266/2522] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:22.632188 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0e81e-88e0-4ccc-aa04-9fcfa3fab911 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:22.646932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0e81e-88e0-4ccc-aa04-9fcfa3fab911 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:22.653317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c6c2f-54be-4f1c-9fc1-d2ecbf3fafe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:22.654290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c6c2f-54be-4f1c-9fc1-d2ecbf3fafe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:22.654654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c6c2f-54be-4f1c-9fc1-d2ecbf3fafe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:22.655466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c6c2f-54be-4f1c-9fc1-d2ecbf3fafe9 admin admin] PATH_INFO: `/v3/projects/2a39401930894645a6fd0aee3497cd16/users/735462efbac1407398eab497d807c0a9/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.680331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c6c2f-54be-4f1c-9fc1-d2ecbf3fafe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a39401930894645a6fd0aee3497cd16, user_id=735462efbac1407398eab497d807c0a9, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:22.683457 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c6c2f-54be-4f1c-9fc1-d2ecbf3fafe9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:22.712458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1257/2523] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:22 2022] PUT /identity/v3/projects/2a39401930894645a6fd0aee3497cd16/users/735462efbac1407398eab497d807c0a9/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:22.718865 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb18c9-e939-41db-8b7b-4a64a51ed6fc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:22.740519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:22.749110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb18c9-e939-41db-8b7b-4a64a51ed6fc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:22.861829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16e51a-0390-4f29-a263-05a9fb9bc4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:22.862337 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16e51a-0390-4f29-a263-05a9fb9bc4be admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:22.864105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16e51a-0390-4f29-a263-05a9fb9bc4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:22.864362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16e51a-0390-4f29-a263-05a9fb9bc4be admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.865185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16e51a-0390-4f29-a263-05a9fb9bc4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:22.867318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16e51a-0390-4f29-a263-05a9fb9bc4be admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:22.883266 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1267/2524] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:22.887436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c6c2f-54be-4f1c-9fc1-d2ecbf3fafe9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:22.900156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c6c2f-54be-4f1c-9fc1-d2ecbf3fafe9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:22.906296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af0d42-8d9e-4e36-8e65-9d4821a88d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:22.906971 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af0d42-8d9e-4e36-8e65-9d4821a88d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:22.907256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af0d42-8d9e-4e36-8e65-9d4821a88d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:22.907530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af0d42-8d9e-4e36-8e65-9d4821a88d77 admin admin] PATH_INFO: `/v3/projects/2a39401930894645a6fd0aee3497cd16/users/735462efbac1407398eab497d807c0a9/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.909509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af0d42-8d9e-4e36-8e65-9d4821a88d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a39401930894645a6fd0aee3497cd16, user_id=735462efbac1407398eab497d807c0a9, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:22.912154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af0d42-8d9e-4e36-8e65-9d4821a88d77 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:22.924053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1258/2525] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:22 2022] PUT /identity/v3/projects/2a39401930894645a6fd0aee3497cd16/users/735462efbac1407398eab497d807c0a9/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:22.935372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c51d-ae46-4d45-bce1-fa601b755de7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:22.935653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c51d-ae46-4d45-bce1-fa601b755de7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:22.935926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c51d-ae46-4d45-bce1-fa601b755de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.973599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1842c51d-ae46-4d45-bce1-fa601b755de7 None None] MFA Rules not processed for user `735462efbac1407398eab497d807c0a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:22.974825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1842c51d-ae46-4d45-bce1-fa601b755de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:23.063117 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1268/2526] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:22 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:23.077637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af0d42-8d9e-4e36-8e65-9d4821a88d77 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:23.079607 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-33af0d42-8d9e-4e36-8e65-9d4821a88d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:23.124259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:23.170746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af0d42-8d9e-4e36-8e65-9d4821a88d77 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:23.176148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950bc67-e73c-4e03-a253-5d38f367795e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:23.176773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950bc67-e73c-4e03-a253-5d38f367795e service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:23.177055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950bc67-e73c-4e03-a253-5d38f367795e service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:23.177357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950bc67-e73c-4e03-a253-5d38f367795e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:23.191803 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950bc67-e73c-4e03-a253-5d38f367795e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:23.193449 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950bc67-e73c-4e03-a253-5d38f367795e service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:23.206595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1259/2527] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:23 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:23.267902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842c51d-ae46-4d45-bce1-fa601b755de7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:23.270396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1842c51d-ae46-4d45-bce1-fa601b755de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:23.285292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842c51d-ae46-4d45-bce1-fa601b755de7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:23.289284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cd766-5551-41bd-bf07-c7521d219164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:23.290163 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd766-5551-41bd-bf07-c7521d219164 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:23.290517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd766-5551-41bd-bf07-c7521d219164 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:23.290854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd766-5551-41bd-bf07-c7521d219164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:23.356184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cd766-5551-41bd-bf07-c7521d219164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:23.360097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cd766-5551-41bd-bf07-c7521d219164 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:23.370302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1269/2528] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:28.174429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950bc67-e73c-4e03-a253-5d38f367795e service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:28.266148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950bc67-e73c-4e03-a253-5d38f367795e service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:28.271920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:28.272721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:28.273055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:28.273383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] PATH_INFO: `/v3/users/2b46cac717084f759335e6ba85863f57` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:28.275076 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b46cac717084f759335e6ba85863f57)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:28.277041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:28.349807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b46cac717084f759335e6ba85863f57'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:28.360592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b46cac717084f759335e6ba85863f57'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:28.361262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b46cac717084f759335e6ba85863f57'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:28.361916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b46cac717084f759335e6ba85863f57'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:28.376987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b46cac717084f759335e6ba85863f57'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:28.395961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1260/2529] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:28 2022] DELETE /identity/v3/users/2b46cac717084f759335e6ba85863f57 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:28.397807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35136082-0587-4828-8871-ae74dbd99350 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:28.398303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35136082-0587-4828-8871-ae74dbd99350 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:28.398424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35136082-0587-4828-8871-ae74dbd99350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:28.421358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-35136082-0587-4828-8871-ae74dbd99350 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:28.422526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-35136082-0587-4828-8871-ae74dbd99350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:28.445369 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:28.505114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1270/2530] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:28.511522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:28.513125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:28.583529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:28.592800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18604d-89ad-40f8-8e39-bfe37b2908d9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:28.598321 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907721f-99a4-4a02-8a92-6a762c12f679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:28.599117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907721f-99a4-4a02-8a92-6a762c12f679 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:28.599957 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907721f-99a4-4a02-8a92-6a762c12f679 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:28.600328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907721f-99a4-4a02-8a92-6a762c12f679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:28.613766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907721f-99a4-4a02-8a92-6a762c12f679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:28.615327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907721f-99a4-4a02-8a92-6a762c12f679 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:28.629195 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1261/2531] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:28.947535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35136082-0587-4828-8871-ae74dbd99350 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:28.949301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-35136082-0587-4828-8871-ae74dbd99350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:28.971951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35136082-0587-4828-8871-ae74dbd99350 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:28.981016 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:28.982331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:28.982621 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:28.982886 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] PATH_INFO: `/v3/projects/aff191159f404c67b2721d2d52208964` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:28.984058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff191159f404c67b2721d2d52208964)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:28.986082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:29.052659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff191159f404c67b2721d2d52208964', 'request_id': 'req-43acab97-240f-4f35-8832-47f55c050415'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.060129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff191159f404c67b2721d2d52208964', 'request_id': 'req-43acab97-240f-4f35-8832-47f55c050415'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.072043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff191159f404c67b2721d2d52208964', 'request_id': 'req-43acab97-240f-4f35-8832-47f55c050415'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.089576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] The token cache is being invalidate because project aff191159f404c67b2721d2d52208964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:29.090248 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.091640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.105808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1271/2532] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:28 2022] DELETE /identity/v3/projects/aff191159f404c67b2721d2d52208964 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:29.108524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907721f-99a4-4a02-8a92-6a762c12f679 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:29.120737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907721f-99a4-4a02-8a92-6a762c12f679 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:29.210088 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:29.210764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:29.211005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:29.211274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] PATH_INFO: `/v3/users/735462efbac1407398eab497d807c0a9` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:29.213225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] RBAC: Authorizing `identity:delete_user(user_id=735462efbac1407398eab497d807c0a9)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:29.214923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:29.274794 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:29.276515 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:29.293606 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acab97-240f-4f35-8832-47f55c050415 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:29.300338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:29.301361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:29.301639 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:29.301909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] PATH_INFO: `/v3/users/a41a18aa9bc543e987e17082f9c990ed` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:29.302899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '735462efbac1407398eab497d807c0a9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.303277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41a18aa9bc543e987e17082f9c990ed)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:29.305988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:29.313521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '735462efbac1407398eab497d807c0a9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.314223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '735462efbac1407398eab497d807c0a9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.314857 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '735462efbac1407398eab497d807c0a9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.334340 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59ea336b-390c-457b-8d68-6b99d04568af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '735462efbac1407398eab497d807c0a9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.355328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1262/2533] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:29 2022] DELETE /identity/v3/users/735462efbac1407398eab497d807c0a9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:29.385358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41a18aa9bc543e987e17082f9c990ed'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.399074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41a18aa9bc543e987e17082f9c990ed'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.409084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41a18aa9bc543e987e17082f9c990ed'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.410008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41a18aa9bc543e987e17082f9c990ed'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.427331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41a18aa9bc543e987e17082f9c990ed'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.440473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1272/2534] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:29 2022] DELETE /identity/v3/users/a41a18aa9bc543e987e17082f9c990ed => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:29.445155 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d25f0-5a9d-475d-9092-9abcaf43bd74 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:29.445652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d25f0-5a9d-475d-9092-9abcaf43bd74 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:29.445922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d25f0-5a9d-475d-9092-9abcaf43bd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:29.471647 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9a5d25f0-5a9d-475d-9092-9abcaf43bd74 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:29.473008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9a5d25f0-5a9d-475d-9092-9abcaf43bd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:29.569333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1263/2535] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:29.581918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:29.583551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:29.618193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:29.674336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6140198-7631-4d22-af09-4b25f1b811dc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:29.678957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a977e-16c4-4b23-9efc-dab033caade1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:29.680273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a977e-16c4-4b23-9efc-dab033caade1 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:29.680407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a977e-16c4-4b23-9efc-dab033caade1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:29.681223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a977e-16c4-4b23-9efc-dab033caade1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:29.701954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a977e-16c4-4b23-9efc-dab033caade1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:29.704054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a977e-16c4-4b23-9efc-dab033caade1 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:29.726925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1273/2536] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:29.728057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d25f0-5a9d-475d-9092-9abcaf43bd74 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:29.730393 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9a5d25f0-5a9d-475d-9092-9abcaf43bd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:29.765877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d25f0-5a9d-475d-9092-9abcaf43bd74 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:29.772554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:29.773052 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:29.773517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:29.773891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] PATH_INFO: `/v3/projects/2a39401930894645a6fd0aee3497cd16` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:29.775296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a39401930894645a6fd0aee3497cd16)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:29.777542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:29.781759 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:29.851312 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a39401930894645a6fd0aee3497cd16', 'request_id': 'req-2dece900-939b-4dbc-a4a3-ded760adf9ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.861342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a39401930894645a6fd0aee3497cd16', 'request_id': 'req-2dece900-939b-4dbc-a4a3-ded760adf9ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.873414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a39401930894645a6fd0aee3497cd16', 'request_id': 'req-2dece900-939b-4dbc-a4a3-ded760adf9ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.893843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] The token cache is being invalidate because project 2a39401930894645a6fd0aee3497cd16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:29.894197 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.894896 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2dece900-939b-4dbc-a4a3-ded760adf9ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:29.897686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1264/2537] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:29 2022] DELETE /identity/v3/projects/2a39401930894645a6fd0aee3497cd16 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:29.902596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328cae2-6141-40d9-8fad-2d7f79b98cfb None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:29.903468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328cae2-6141-40d9-8fad-2d7f79b98cfb None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:29.903468 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328cae2-6141-40d9-8fad-2d7f79b98cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:29.934210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b328cae2-6141-40d9-8fad-2d7f79b98cfb None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:29.936898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b328cae2-6141-40d9-8fad-2d7f79b98cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:30.046974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1274/2538] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:30.054887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b291628-89a2-4333-a749-34eed7ea8156 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.055502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b291628-89a2-4333-a749-34eed7ea8156 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.056041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b291628-89a2-4333-a749-34eed7ea8156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.098375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9b291628-89a2-4333-a749-34eed7ea8156 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:30.099653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9b291628-89a2-4333-a749-34eed7ea8156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:30.119020 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1265/2539] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:30.121936 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328cae2-6141-40d9-8fad-2d7f79b98cfb None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:30.143534 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b291628-89a2-4333-a749-34eed7ea8156 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:30.147805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328cae2-6141-40d9-8fad-2d7f79b98cfb None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:30.148727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9b291628-89a2-4333-a749-34eed7ea8156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:30.153036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a7fb4-0876-49dc-a82b-24346f0ebfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:30.158183 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a7fb4-0876-49dc-a82b-24346f0ebfed admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.158603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a7fb4-0876-49dc-a82b-24346f0ebfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.158976 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a7fb4-0876-49dc-a82b-24346f0ebfed admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.159812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a7fb4-0876-49dc-a82b-24346f0ebfed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:30.160224 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a7fb4-0876-49dc-a82b-24346f0ebfed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:30.162077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a7fb4-0876-49dc-a82b-24346f0ebfed admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:30.170931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b291628-89a2-4333-a749-34eed7ea8156 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:30.178130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:30.178848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.179127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.179397 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] PATH_INFO: `/v3/projects/dd28b75bd4644ebf9012e68aae8c3fb9` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.181204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd28b75bd4644ebf9012e68aae8c3fb9)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:30.183426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:30.192716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1275/2540] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:30.193885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a7fb4-0876-49dc-a82b-24346f0ebfed admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:30.205995 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a7fb4-0876-49dc-a82b-24346f0ebfed admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:30.212774 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220eb6a-5e58-4bd3-9316-f626da6ad1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:30.213614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220eb6a-5e58-4bd3-9316-f626da6ad1ac admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.213972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220eb6a-5e58-4bd3-9316-f626da6ad1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.214316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220eb6a-5e58-4bd3-9316-f626da6ad1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.215053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220eb6a-5e58-4bd3-9316-f626da6ad1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:30.219110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220eb6a-5e58-4bd3-9316-f626da6ad1ac admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:30.290064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1276/2541] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:30 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:30.290064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220eb6a-5e58-4bd3-9316-f626da6ad1ac admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:30.321606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd28b75bd4644ebf9012e68aae8c3fb9', 'request_id': 'req-bd76cc99-36d4-49d4-8b30-72967c652bd9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.330242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd28b75bd4644ebf9012e68aae8c3fb9', 'request_id': 'req-bd76cc99-36d4-49d4-8b30-72967c652bd9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.342608 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd28b75bd4644ebf9012e68aae8c3fb9', 'request_id': 'req-bd76cc99-36d4-49d4-8b30-72967c652bd9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.360452 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] The token cache is being invalidate because project dd28b75bd4644ebf9012e68aae8c3fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:30.360697 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.361473 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.366844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1266/2542] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:30 2022] DELETE /identity/v3/projects/dd28b75bd4644ebf9012e68aae8c3fb9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:30.374786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:30.376507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:30.395584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220eb6a-5e58-4bd3-9316-f626da6ad1ac admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:30.402343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189745aa-e903-4028-b92d-46c89cfc174e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:30.402981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189745aa-e903-4028-b92d-46c89cfc174e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.403251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189745aa-e903-4028-b92d-46c89cfc174e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.403538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189745aa-e903-4028-b92d-46c89cfc174e admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.404263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189745aa-e903-4028-b92d-46c89cfc174e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:30.407320 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189745aa-e903-4028-b92d-46c89cfc174e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:30.437695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1277/2543] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:30 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:30.442372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189745aa-e903-4028-b92d-46c89cfc174e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:30.445376 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-189745aa-e903-4028-b92d-46c89cfc174e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:30.458900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189745aa-e903-4028-b92d-46c89cfc174e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:30.467944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccebda9-e247-4fb6-aa38-8c59514dc07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:30.468769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccebda9-e247-4fb6-aa38-8c59514dc07a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.469049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccebda9-e247-4fb6-aa38-8c59514dc07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.469341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccebda9-e247-4fb6-aa38-8c59514dc07a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.470511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccebda9-e247-4fb6-aa38-8c59514dc07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:30.472401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccebda9-e247-4fb6-aa38-8c59514dc07a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:30.484684 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1278/2544] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:30.485236 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76cc99-36d4-49d4-8b30-72967c652bd9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:30.489455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccebda9-e247-4fb6-aa38-8c59514dc07a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:30.490746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:30.491505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.491858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.492317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] PATH_INFO: `/v3/users/c08fb7c89fb64799ac13e421f83f655d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.493563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08fb7c89fb64799ac13e421f83f655d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:30.495562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:30.501837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccebda9-e247-4fb6-aa38-8c59514dc07a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:30.511361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:30.513944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fec0-bfa4-4e74-9546-54e1217a2acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:30.515009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fec0-bfa4-4e74-9546-54e1217a2acc admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.516071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fec0-bfa4-4e74-9546-54e1217a2acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.516071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fec0-bfa4-4e74-9546-54e1217a2acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.516956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1fec0-bfa4-4e74-9546-54e1217a2acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:30.518804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1fec0-bfa4-4e74-9546-54e1217a2acc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:30.520553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:30.541760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1279/2545] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:30.543547 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fec0-bfa4-4e74-9546-54e1217a2acc admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:30.560703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fec0-bfa4-4e74-9546-54e1217a2acc admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:30.567435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad835a6-b6e1-4900-97b8-a93ca4f1b9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:30.568288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08fb7c89fb64799ac13e421f83f655d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.568416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad835a6-b6e1-4900-97b8-a93ca4f1b9c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.568587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad835a6-b6e1-4900-97b8-a93ca4f1b9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.568843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad835a6-b6e1-4900-97b8-a93ca4f1b9c9 admin admin] PATH_INFO: `/v3/projects/d6e9c4e4fe8f453d9c56986296a22571/users/5818559dc01341cb9bca172ee6391eef/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.583049 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08fb7c89fb64799ac13e421f83f655d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.584291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08fb7c89fb64799ac13e421f83f655d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.584923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08fb7c89fb64799ac13e421f83f655d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.594313 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad835a6-b6e1-4900-97b8-a93ca4f1b9c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e9c4e4fe8f453d9c56986296a22571, user_id=5818559dc01341cb9bca172ee6391eef, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:30.597505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad835a6-b6e1-4900-97b8-a93ca4f1b9c9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:30.610165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08fb7c89fb64799ac13e421f83f655d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:30.615280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1280/2546] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:30 2022] PUT /identity/v3/projects/d6e9c4e4fe8f453d9c56986296a22571/users/5818559dc01341cb9bca172ee6391eef/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:30.627999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a793-9e5b-4af4-a353-8fa67ac38584 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.629206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a793-9e5b-4af4-a353-8fa67ac38584 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.629499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a793-9e5b-4af4-a353-8fa67ac38584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.640256 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1267/2547] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:30 2022] DELETE /identity/v3/users/c08fb7c89fb64799ac13e421f83f655d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:30.678924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-4ea0a793-9e5b-4af4-a353-8fa67ac38584 None None] MFA Rules not processed for user `5818559dc01341cb9bca172ee6391eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:30.680737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4ea0a793-9e5b-4af4-a353-8fa67ac38584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:30.834852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1281/2548] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:30.850156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:30.851877 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:30.937969 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58463669-5759-44d3-a677-1e84a3d32c32 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:30.942109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6b366-9f14-434d-bc18-30ba95f2ecc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:30.942727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6b366-9f14-434d-bc18-30ba95f2ecc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.943003 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6b366-9f14-434d-bc18-30ba95f2ecc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.943273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6b366-9f14-434d-bc18-30ba95f2ecc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.953191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6b366-9f14-434d-bc18-30ba95f2ecc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:30.954629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6b366-9f14-434d-bc18-30ba95f2ecc6 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:30.964392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1268/2549] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:31.022669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0a793-9e5b-4af4-a353-8fa67ac38584 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.024093 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-4ea0a793-9e5b-4af4-a353-8fa67ac38584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:31.119448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0a793-9e5b-4af4-a353-8fa67ac38584 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:31.125346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:31.125993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.126407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.126681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] PATH_INFO: `/v3/projects/72377d04b72c4c0aaf2bdff9321039a8` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.127874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72377d04b72c4c0aaf2bdff9321039a8)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:31.130771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:31.186674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72377d04b72c4c0aaf2bdff9321039a8', 'request_id': 'req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:31.194353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72377d04b72c4c0aaf2bdff9321039a8', 'request_id': 'req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:31.204152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72377d04b72c4c0aaf2bdff9321039a8', 'request_id': 'req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:31.206968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:31.220243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] The token cache is being invalidate because project 72377d04b72c4c0aaf2bdff9321039a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:31.220542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:31.221732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-2bad51e9-e69f-4b6e-9b05-7af1761cfd24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:31.224214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1282/2550] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:31 2022] DELETE /identity/v3/projects/72377d04b72c4c0aaf2bdff9321039a8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:31.229376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bf41a-de76-4706-b78f-d47b5e64e27d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.229975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bf41a-de76-4706-b78f-d47b5e64e27d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.230250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bf41a-de76-4706-b78f-d47b5e64e27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.256023 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1d5bf41a-de76-4706-b78f-d47b5e64e27d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:31.257656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1d5bf41a-de76-4706-b78f-d47b5e64e27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:31.357932 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1269/2551] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:31.365741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240271d2-4bfb-4398-bff6-65e48273e6e5 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.366053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240271d2-4bfb-4398-bff6-65e48273e6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.368620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240271d2-4bfb-4398-bff6-65e48273e6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.391423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-240271d2-4bfb-4398-bff6-65e48273e6e5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:31.395459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-240271d2-4bfb-4398-bff6-65e48273e6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:31.410740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1283/2552] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:31.414880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bf41a-de76-4706-b78f-d47b5e64e27d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.425972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bf41a-de76-4706-b78f-d47b5e64e27d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:31.430952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ddaa1-e63c-446e-a59d-50683bc4eb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:31.431840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ddaa1-e63c-446e-a59d-50683bc4eb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.432304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ddaa1-e63c-446e-a59d-50683bc4eb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.432579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ddaa1-e63c-446e-a59d-50683bc4eb81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.433348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ddaa1-e63c-446e-a59d-50683bc4eb81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:31.433569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ddaa1-e63c-446e-a59d-50683bc4eb81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:31.437245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ddaa1-e63c-446e-a59d-50683bc4eb81 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:31.454816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1270/2553] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:08:31.459584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240271d2-4bfb-4398-bff6-65e48273e6e5 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.470541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240271d2-4bfb-4398-bff6-65e48273e6e5 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:31.475596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e027b-7c54-4872-801f-47f5ad164d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:31.476206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e027b-7c54-4872-801f-47f5ad164d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.476484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e027b-7c54-4872-801f-47f5ad164d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.476749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e027b-7c54-4872-801f-47f5ad164d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.477418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e027b-7c54-4872-801f-47f5ad164d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:31.479315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e027b-7c54-4872-801f-47f5ad164d3a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:31.510061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1284/2554] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:31 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:31.514979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ddaa1-e63c-446e-a59d-50683bc4eb81 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.524902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ddaa1-e63c-446e-a59d-50683bc4eb81 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:31.537544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:31.583285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b65cb-8f62-4846-878a-b5e5593354ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:31.583811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b65cb-8f62-4846-878a-b5e5593354ab admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.584119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b65cb-8f62-4846-878a-b5e5593354ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.584392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b65cb-8f62-4846-878a-b5e5593354ab admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.585025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b65cb-8f62-4846-878a-b5e5593354ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:31.586959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b65cb-8f62-4846-878a-b5e5593354ab admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:31.607618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1271/2555] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:31 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:31.611727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e027b-7c54-4872-801f-47f5ad164d3a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.622472 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e027b-7c54-4872-801f-47f5ad164d3a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:31.627024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90f306-c242-4473-a3f7-7dce1433e2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:31.627610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90f306-c242-4473-a3f7-7dce1433e2af admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.627880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90f306-c242-4473-a3f7-7dce1433e2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.628142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90f306-c242-4473-a3f7-7dce1433e2af admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.628769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90f306-c242-4473-a3f7-7dce1433e2af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:31.630390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90f306-c242-4473-a3f7-7dce1433e2af admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:31.640822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1285/2556] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:31.645357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b65cb-8f62-4846-878a-b5e5593354ab admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.655895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b65cb-8f62-4846-878a-b5e5593354ab admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:31.660561 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e978521-ad3f-4c22-a654-9933d9ad75e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:31.661179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e978521-ad3f-4c22-a654-9933d9ad75e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.661458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e978521-ad3f-4c22-a654-9933d9ad75e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.661725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e978521-ad3f-4c22-a654-9933d9ad75e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.662357 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e978521-ad3f-4c22-a654-9933d9ad75e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:31.663907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e978521-ad3f-4c22-a654-9933d9ad75e4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:31.675242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1272/2557] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:31.679193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90f306-c242-4473-a3f7-7dce1433e2af admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.689942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90f306-c242-4473-a3f7-7dce1433e2af admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:31.694637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec02cd-6fcf-46f9-a75d-d02a12bad8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:31.695274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec02cd-6fcf-46f9-a75d-d02a12bad8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.695553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec02cd-6fcf-46f9-a75d-d02a12bad8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.695824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec02cd-6fcf-46f9-a75d-d02a12bad8bf admin admin] PATH_INFO: `/v3/projects/6cab18384cab4aacb82320a4b3af8ded/users/950ef77a2d294a839f3baa47107aab4c/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.712769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec02cd-6fcf-46f9-a75d-d02a12bad8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cab18384cab4aacb82320a4b3af8ded, user_id=950ef77a2d294a839f3baa47107aab4c, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:31.715437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec02cd-6fcf-46f9-a75d-d02a12bad8bf admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:31.728220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1286/2558] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:31 2022] PUT /identity/v3/projects/6cab18384cab4aacb82320a4b3af8ded/users/950ef77a2d294a839f3baa47107aab4c/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:31.739914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28566d-4c11-48a9-8489-c5034a0c8ec5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.740250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28566d-4c11-48a9-8489-c5034a0c8ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.740521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28566d-4c11-48a9-8489-c5034a0c8ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.770244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5b28566d-4c11-48a9-8489-c5034a0c8ec5 None None] MFA Rules not processed for user `950ef77a2d294a839f3baa47107aab4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:31.771471 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5b28566d-4c11-48a9-8489-c5034a0c8ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:31.837400 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1273/2559] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:31.842441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec02cd-6fcf-46f9-a75d-d02a12bad8bf admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.902398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec02cd-6fcf-46f9-a75d-d02a12bad8bf admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:31.908262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81716ec-0039-4509-9c42-aff55af4ba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:31.908926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81716ec-0039-4509-9c42-aff55af4ba17 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.909240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81716ec-0039-4509-9c42-aff55af4ba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.909508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81716ec-0039-4509-9c42-aff55af4ba17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.910142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81716ec-0039-4509-9c42-aff55af4ba17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:31.912131 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81716ec-0039-4509-9c42-aff55af4ba17 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:31.941348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1287/2560] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:31 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:31.945516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28566d-4c11-48a9-8489-c5034a0c8ec5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.956556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28566d-4c11-48a9-8489-c5034a0c8ec5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:32.013599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a61401-8fae-4b85-9ca8-eadc6b121c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:32.014204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a61401-8fae-4b85-9ca8-eadc6b121c7f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.014478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a61401-8fae-4b85-9ca8-eadc6b121c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.014744 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a61401-8fae-4b85-9ca8-eadc6b121c7f admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.015366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a61401-8fae-4b85-9ca8-eadc6b121c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:32.017266 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a61401-8fae-4b85-9ca8-eadc6b121c7f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:32.039987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1274/2561] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:31 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:32.044648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81716ec-0039-4509-9c42-aff55af4ba17 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:32.055040 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81716ec-0039-4509-9c42-aff55af4ba17 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:32.059709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c296f6d-b4a5-4f1c-b1a4-5abb40ecddb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:32.060314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c296f6d-b4a5-4f1c-b1a4-5abb40ecddb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.060591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c296f6d-b4a5-4f1c-b1a4-5abb40ecddb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.060861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c296f6d-b4a5-4f1c-b1a4-5abb40ecddb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.061536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c296f6d-b4a5-4f1c-b1a4-5abb40ecddb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:32.063134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c296f6d-b4a5-4f1c-b1a4-5abb40ecddb3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:32.073857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1288/2562] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:32.078348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a61401-8fae-4b85-9ca8-eadc6b121c7f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:32.088500 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a61401-8fae-4b85-9ca8-eadc6b121c7f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:32.093979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d10c3c-6dde-4102-8993-520b1073752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:32.094632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d10c3c-6dde-4102-8993-520b1073752c admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.094910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d10c3c-6dde-4102-8993-520b1073752c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.095204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d10c3c-6dde-4102-8993-520b1073752c admin admin] PATH_INFO: `/v3/projects/8513c3f13be24f3aafe21b94b704f44b/users/c230f15fe4504e718af01e1fde5de851/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.108066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:32.111477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d10c3c-6dde-4102-8993-520b1073752c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8513c3f13be24f3aafe21b94b704f44b, user_id=c230f15fe4504e718af01e1fde5de851, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:32.114788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d10c3c-6dde-4102-8993-520b1073752c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:32.128491 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1275/2563] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:32 2022] PUT /identity/v3/projects/8513c3f13be24f3aafe21b94b704f44b/users/c230f15fe4504e718af01e1fde5de851/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:32.132593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c296f6d-b4a5-4f1c-b1a4-5abb40ecddb3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:32.143318 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c296f6d-b4a5-4f1c-b1a4-5abb40ecddb3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:32.202346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7b504-408e-4b73-bf85-53923a0be7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:32.202951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7b504-408e-4b73-bf85-53923a0be7fa admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.203227 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7b504-408e-4b73-bf85-53923a0be7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.203492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7b504-408e-4b73-bf85-53923a0be7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.204130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7b504-408e-4b73-bf85-53923a0be7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:32.205787 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7b504-408e-4b73-bf85-53923a0be7fa admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:32.207966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:32.217793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1289/2564] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:32.221782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d10c3c-6dde-4102-8993-520b1073752c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:32.232914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d10c3c-6dde-4102-8993-520b1073752c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:32.238471 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fded43-9cf9-475c-9c41-97b246195177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:32.239137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fded43-9cf9-475c-9c41-97b246195177 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.239439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fded43-9cf9-475c-9c41-97b246195177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.239698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fded43-9cf9-475c-9c41-97b246195177 admin admin] PATH_INFO: `/v3/projects/8513c3f13be24f3aafe21b94b704f44b/users/c230f15fe4504e718af01e1fde5de851/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.241677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fded43-9cf9-475c-9c41-97b246195177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8513c3f13be24f3aafe21b94b704f44b, user_id=c230f15fe4504e718af01e1fde5de851, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:32.244409 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fded43-9cf9-475c-9c41-97b246195177 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:32.258430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1276/2565] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:32 2022] PUT /identity/v3/projects/8513c3f13be24f3aafe21b94b704f44b/users/c230f15fe4504e718af01e1fde5de851/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:32.270390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5744778-7ca9-4f7e-bb37-4e76a9ce3310 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.270806 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5744778-7ca9-4f7e-bb37-4e76a9ce3310 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.271154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5744778-7ca9-4f7e-bb37-4e76a9ce3310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.299862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e5744778-7ca9-4f7e-bb37-4e76a9ce3310 None None] MFA Rules not processed for user `c230f15fe4504e718af01e1fde5de851`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:32.301110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e5744778-7ca9-4f7e-bb37-4e76a9ce3310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:32.382132 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1290/2566] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:32.390311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fded43-9cf9-475c-9c41-97b246195177 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:32.392791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c2fded43-9cf9-475c-9c41-97b246195177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:32.469284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fded43-9cf9-475c-9c41-97b246195177 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:32.473271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56edf94-0cd5-40fc-9696-5fabc347c6ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:32.473894 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56edf94-0cd5-40fc-9696-5fabc347c6ef service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.474174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56edf94-0cd5-40fc-9696-5fabc347c6ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.474434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56edf94-0cd5-40fc-9696-5fabc347c6ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.488807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56edf94-0cd5-40fc-9696-5fabc347c6ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:32.490320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56edf94-0cd5-40fc-9696-5fabc347c6ef service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:32.501366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1277/2567] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:32.534452 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5744778-7ca9-4f7e-bb37-4e76a9ce3310 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:32.607303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5744778-7ca9-4f7e-bb37-4e76a9ce3310 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:32.612617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:32.613324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.613607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.613898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] PATH_INFO: `/v3/users/950ef77a2d294a839f3baa47107aab4c` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.615166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=950ef77a2d294a839f3baa47107aab4c)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:32.617141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:32.680878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950ef77a2d294a839f3baa47107aab4c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:32.690231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950ef77a2d294a839f3baa47107aab4c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:32.697946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950ef77a2d294a839f3baa47107aab4c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:32.698584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950ef77a2d294a839f3baa47107aab4c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:32.711692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950ef77a2d294a839f3baa47107aab4c'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:32.718742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1291/2568] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:32 2022] DELETE /identity/v3/users/950ef77a2d294a839f3baa47107aab4c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:32.723308 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e2f34-fd35-4494-afc7-25269cd0fb65 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.723591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e2f34-fd35-4494-afc7-25269cd0fb65 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.723860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e2f34-fd35-4494-afc7-25269cd0fb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.742425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2b3e2f34-fd35-4494-afc7-25269cd0fb65 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:32.743616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2b3e2f34-fd35-4494-afc7-25269cd0fb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:32.803719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1278/2569] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:32.814186 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:32.815585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:32.820899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:32.875314 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f12646-0589-40a6-af9b-3f8770c06051 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:32.879001 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05ef34-197e-44f9-880c-79b75143ea4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:32.879600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ef34-197e-44f9-880c-79b75143ea4d service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:32.879879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ef34-197e-44f9-880c-79b75143ea4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:32.880148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ef34-197e-44f9-880c-79b75143ea4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:32.891297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05ef34-197e-44f9-880c-79b75143ea4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:32.895044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05ef34-197e-44f9-880c-79b75143ea4d service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:32.907926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1292/2570] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:33.176727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e2f34-fd35-4494-afc7-25269cd0fb65 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:33.178187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2b3e2f34-fd35-4494-afc7-25269cd0fb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:33.190347 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e2f34-fd35-4494-afc7-25269cd0fb65 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:33.195090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:33.195714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:33.195993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:33.196264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] PATH_INFO: `/v3/projects/6cab18384cab4aacb82320a4b3af8ded` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:33.197415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cab18384cab4aacb82320a4b3af8ded)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:33.199292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:33.245836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cab18384cab4aacb82320a4b3af8ded', 'request_id': 'req-738526be-f549-4073-b6a3-d1fb76cc5092'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.252090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cab18384cab4aacb82320a4b3af8ded', 'request_id': 'req-738526be-f549-4073-b6a3-d1fb76cc5092'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.262992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cab18384cab4aacb82320a4b3af8ded', 'request_id': 'req-738526be-f549-4073-b6a3-d1fb76cc5092'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.274057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] The token cache is being invalidate because project 6cab18384cab4aacb82320a4b3af8ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:33.274356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.274995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.276656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1279/2571] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:33 2022] DELETE /identity/v3/projects/6cab18384cab4aacb82320a4b3af8ded => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:33.281572 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05ef34-197e-44f9-880c-79b75143ea4d service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:33.291289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05ef34-197e-44f9-880c-79b75143ea4d service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:33.347671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:33.348293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:33.348574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:33.348848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] PATH_INFO: `/v3/users/c230f15fe4504e718af01e1fde5de851` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:33.350002 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] RBAC: Authorizing `identity:delete_user(user_id=c230f15fe4504e718af01e1fde5de851)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:33.351918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:33.397126 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c230f15fe4504e718af01e1fde5de851'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.404151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c230f15fe4504e718af01e1fde5de851'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.411365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c230f15fe4504e718af01e1fde5de851'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.411974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c230f15fe4504e718af01e1fde5de851'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.423993 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-16bbf623-4548-4504-b15c-6518e62550af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c230f15fe4504e718af01e1fde5de851'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.429670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1293/2572] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:33 2022] DELETE /identity/v3/users/c230f15fe4504e718af01e1fde5de851 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:33.709884 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:33.711356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:33.722382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:33.727626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738526be-f549-4073-b6a3-d1fb76cc5092 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:33.793650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:33.794959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:33.795211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:33.795387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] PATH_INFO: `/v3/projects/8513c3f13be24f3aafe21b94b704f44b` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:33.796698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8513c3f13be24f3aafe21b94b704f44b)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:33.798698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:33.848562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8513c3f13be24f3aafe21b94b704f44b', 'request_id': 'req-d329adad-a463-4016-b2a4-6c0598c86e1f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.854818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8513c3f13be24f3aafe21b94b704f44b', 'request_id': 'req-d329adad-a463-4016-b2a4-6c0598c86e1f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.863253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8513c3f13be24f3aafe21b94b704f44b', 'request_id': 'req-d329adad-a463-4016-b2a4-6c0598c86e1f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.874965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] The token cache is being invalidate because project 8513c3f13be24f3aafe21b94b704f44b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:33.875251 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.875922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d329adad-a463-4016-b2a4-6c0598c86e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:33.877691 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1280/2573] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:33 2022] DELETE /identity/v3/projects/8513c3f13be24f3aafe21b94b704f44b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:33.882853 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d549b8f-c415-4b6f-b162-8408b7136326 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:33.883308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d549b8f-c415-4b6f-b162-8408b7136326 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:33.883546 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d549b8f-c415-4b6f-b162-8408b7136326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:33.904698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9d549b8f-c415-4b6f-b162-8408b7136326 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:33.905920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9d549b8f-c415-4b6f-b162-8408b7136326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:33.924728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:33.966703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1294/2574] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:33.974418 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97630862-14d6-4c21-a3b5-60079d72b25a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:33.974704 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97630862-14d6-4c21-a3b5-60079d72b25a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:33.975005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97630862-14d6-4c21-a3b5-60079d72b25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:33.995349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-97630862-14d6-4c21-a3b5-60079d72b25a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:33.996554 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-97630862-14d6-4c21-a3b5-60079d72b25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:34.005386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1281/2575] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:34.009531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d549b8f-c415-4b6f-b162-8408b7136326 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:34.019388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d549b8f-c415-4b6f-b162-8408b7136326 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:34.023913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ffe80-db8f-45be-98f2-82ca8e3c0108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:34.024497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ffe80-db8f-45be-98f2-82ca8e3c0108 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.024770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ffe80-db8f-45be-98f2-82ca8e3c0108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.025032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ffe80-db8f-45be-98f2-82ca8e3c0108 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.025701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ffe80-db8f-45be-98f2-82ca8e3c0108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:34.025997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ffe80-db8f-45be-98f2-82ca8e3c0108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.027611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ffe80-db8f-45be-98f2-82ca8e3c0108 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.040159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1295/2576] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:34.044916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97630862-14d6-4c21-a3b5-60079d72b25a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:34.053980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97630862-14d6-4c21-a3b5-60079d72b25a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:34.060245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eb663-d575-4186-9c06-87f990b8348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:34.061037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eb663-d575-4186-9c06-87f990b8348f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.061344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eb663-d575-4186-9c06-87f990b8348f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.061614 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eb663-d575-4186-9c06-87f990b8348f admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.062237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eb663-d575-4186-9c06-87f990b8348f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.064111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eb663-d575-4186-9c06-87f990b8348f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.095812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1282/2577] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:34 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:34.096971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ffe80-db8f-45be-98f2-82ca8e3c0108 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:34.107379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ffe80-db8f-45be-98f2-82ca8e3c0108 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:34.165620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a22b8-31d7-4820-9115-7332c733fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:34.166232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a22b8-31d7-4820-9115-7332c733fae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.166494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a22b8-31d7-4820-9115-7332c733fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.166760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a22b8-31d7-4820-9115-7332c733fae9 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.167379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a22b8-31d7-4820-9115-7332c733fae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.169473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a22b8-31d7-4820-9115-7332c733fae9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.190858 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1296/2578] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:34 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:34.194899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eb663-d575-4186-9c06-87f990b8348f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:34.205889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eb663-d575-4186-9c06-87f990b8348f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:34.210456 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53142c72-76e0-47ca-9eb2-34e7e783f199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:34.211041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53142c72-76e0-47ca-9eb2-34e7e783f199 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.211315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53142c72-76e0-47ca-9eb2-34e7e783f199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.211582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53142c72-76e0-47ca-9eb2-34e7e783f199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.212213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53142c72-76e0-47ca-9eb2-34e7e783f199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.213809 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53142c72-76e0-47ca-9eb2-34e7e783f199 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.224311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1283/2579] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:34.228127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a22b8-31d7-4820-9115-7332c733fae9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:34.238463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a22b8-31d7-4820-9115-7332c733fae9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:34.243145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cea4da-f1c5-4560-b614-d2ac53091f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:34.243747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cea4da-f1c5-4560-b614-d2ac53091f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.244031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cea4da-f1c5-4560-b614-d2ac53091f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.244293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cea4da-f1c5-4560-b614-d2ac53091f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.244929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cea4da-f1c5-4560-b614-d2ac53091f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.246560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cea4da-f1c5-4560-b614-d2ac53091f95 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.257652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1297/2580] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:34.261869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53142c72-76e0-47ca-9eb2-34e7e783f199 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:34.272586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53142c72-76e0-47ca-9eb2-34e7e783f199 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:34.277117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81df07-b4b3-48d1-8f4b-1475c65a423b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:34.277754 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81df07-b4b3-48d1-8f4b-1475c65a423b admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.278684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81df07-b4b3-48d1-8f4b-1475c65a423b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.278956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81df07-b4b3-48d1-8f4b-1475c65a423b admin admin] PATH_INFO: `/v3/projects/41cb4dd000c846fabffb338ded81970a/users/dc7789b8d146414897c5fc970d6e8817/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.293210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81df07-b4b3-48d1-8f4b-1475c65a423b admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cb4dd000c846fabffb338ded81970a, user_id=dc7789b8d146414897c5fc970d6e8817, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.295907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81df07-b4b3-48d1-8f4b-1475c65a423b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.308223 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1284/2581] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:34 2022] PUT /identity/v3/projects/41cb4dd000c846fabffb338ded81970a/users/dc7789b8d146414897c5fc970d6e8817/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:34.319041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e327b8-c9f9-4614-8226-0946017d971e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.319327 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e327b8-c9f9-4614-8226-0946017d971e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.319600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e327b8-c9f9-4614-8226-0946017d971e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.348316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-41e327b8-c9f9-4614-8226-0946017d971e None None] MFA Rules not processed for user `dc7789b8d146414897c5fc970d6e8817`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:34.349593 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-41e327b8-c9f9-4614-8226-0946017d971e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:34.412540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1298/2582] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:34.423667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81df07-b4b3-48d1-8f4b-1475c65a423b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:34.425381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fb81df07-b4b3-48d1-8f4b-1475c65a423b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:34.460296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:34.484690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81df07-b4b3-48d1-8f4b-1475c65a423b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:34.488258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f2852-0d39-4906-8076-cc9b89fea845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:34.488857 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f2852-0d39-4906-8076-cc9b89fea845 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.489179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f2852-0d39-4906-8076-cc9b89fea845 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.489454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f2852-0d39-4906-8076-cc9b89fea845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.498373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f2852-0d39-4906-8076-cc9b89fea845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.499788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f2852-0d39-4906-8076-cc9b89fea845 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.508467 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1285/2583] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:34.530598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e327b8-c9f9-4614-8226-0946017d971e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:34.551780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:34.594959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e327b8-c9f9-4614-8226-0946017d971e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:34.599570 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:34.600231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.600510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.600777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] PATH_INFO: `/v3/users/dc7789b8d146414897c5fc970d6e8817` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.601936 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc7789b8d146414897c5fc970d6e8817)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.603866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.649162 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc7789b8d146414897c5fc970d6e8817'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:34.656536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc7789b8d146414897c5fc970d6e8817'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:34.663374 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc7789b8d146414897c5fc970d6e8817'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:34.664011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc7789b8d146414897c5fc970d6e8817'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:34.676585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc7789b8d146414897c5fc970d6e8817'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:34.682390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1299/2584] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:34 2022] DELETE /identity/v3/users/dc7789b8d146414897c5fc970d6e8817 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:34.688146 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17f07-fbaf-4086-944f-821643ce21b1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.688430 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17f07-fbaf-4086-944f-821643ce21b1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.688700 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17f07-fbaf-4086-944f-821643ce21b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.708477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-77d17f07-fbaf-4086-944f-821643ce21b1 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:34.709707 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-77d17f07-fbaf-4086-944f-821643ce21b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:34.769239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1286/2585] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:34.779859 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:34.781287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:34.838956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e6563-0053-4314-bebc-950ce25b7e45 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:34.843713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3822b29-5ac6-484a-9850-d71ce441f523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:34.844310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3822b29-5ac6-484a-9850-d71ce441f523 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:34.844584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3822b29-5ac6-484a-9850-d71ce441f523 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:34.844849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3822b29-5ac6-484a-9850-d71ce441f523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:34.853880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3822b29-5ac6-484a-9850-d71ce441f523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.855305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3822b29-5ac6-484a-9850-d71ce441f523 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.865358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1300/2586] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:08:35.190977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d17f07-fbaf-4086-944f-821643ce21b1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:35.192427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-77d17f07-fbaf-4086-944f-821643ce21b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:35.205566 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d17f07-fbaf-4086-944f-821643ce21b1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:35.210375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:35.210984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.211264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.211535 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] PATH_INFO: `/v3/projects/41cb4dd000c846fabffb338ded81970a` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.212657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41cb4dd000c846fabffb338ded81970a)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:35.214599 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:35.271693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41cb4dd000c846fabffb338ded81970a', 'request_id': 'req-576150df-7cee-4644-97a1-1fb587799a76'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:35.277891 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41cb4dd000c846fabffb338ded81970a', 'request_id': 'req-576150df-7cee-4644-97a1-1fb587799a76'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:35.286257 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41cb4dd000c846fabffb338ded81970a', 'request_id': 'req-576150df-7cee-4644-97a1-1fb587799a76'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:35.296917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] The token cache is being invalidate because project 41cb4dd000c846fabffb338ded81970a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:35.297183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:35.297822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-576150df-7cee-4644-97a1-1fb587799a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:35.299311 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1287/2587] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:35 2022] DELETE /identity/v3/projects/41cb4dd000c846fabffb338ded81970a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:35.304140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935ec55-4100-41b4-82ad-53b3c77b0649 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.304628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935ec55-4100-41b4-82ad-53b3c77b0649 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.304735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935ec55-4100-41b4-82ad-53b3c77b0649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.324330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d935ec55-4100-41b4-82ad-53b3c77b0649 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:35.326284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d935ec55-4100-41b4-82ad-53b3c77b0649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:35.363304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:35.385599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1301/2588] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:35.395368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224ae19-e693-4720-ae6d-17c62ca38878 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.395645 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224ae19-e693-4720-ae6d-17c62ca38878 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.395938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224ae19-e693-4720-ae6d-17c62ca38878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.415997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8224ae19-e693-4720-ae6d-17c62ca38878 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:35.417207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8224ae19-e693-4720-ae6d-17c62ca38878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:35.425753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1288/2589] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:35.429924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935ec55-4100-41b4-82ad-53b3c77b0649 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:35.440291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935ec55-4100-41b4-82ad-53b3c77b0649 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:35.444877 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66812000-af3c-4a00-af53-48d7f23fff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:35.445514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66812000-af3c-4a00-af53-48d7f23fff72 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.445798 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66812000-af3c-4a00-af53-48d7f23fff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.446071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66812000-af3c-4a00-af53-48d7f23fff72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.446709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66812000-af3c-4a00-af53-48d7f23fff72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:35.447010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66812000-af3c-4a00-af53-48d7f23fff72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:35.448590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66812000-af3c-4a00-af53-48d7f23fff72 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:35.459978 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1302/2590] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:35.464111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224ae19-e693-4720-ae6d-17c62ca38878 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:35.473663 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224ae19-e693-4720-ae6d-17c62ca38878 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:35.478315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f33768-f8fd-4543-ac55-cbc860adc887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:35.478899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f33768-f8fd-4543-ac55-cbc860adc887 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.479171 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f33768-f8fd-4543-ac55-cbc860adc887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.479435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f33768-f8fd-4543-ac55-cbc860adc887 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.480051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f33768-f8fd-4543-ac55-cbc860adc887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:35.482605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f33768-f8fd-4543-ac55-cbc860adc887 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:35.511529 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1289/2591] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:35 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:35.516506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66812000-af3c-4a00-af53-48d7f23fff72 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:35.526354 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66812000-af3c-4a00-af53-48d7f23fff72 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:35.586018 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd13c6-87ca-4f4f-afe6-2de7ea648752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:35.586538 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd13c6-87ca-4f4f-afe6-2de7ea648752 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.586816 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd13c6-87ca-4f4f-afe6-2de7ea648752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.587085 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd13c6-87ca-4f4f-afe6-2de7ea648752 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.587707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd13c6-87ca-4f4f-afe6-2de7ea648752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:35.589653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd13c6-87ca-4f4f-afe6-2de7ea648752 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:35.614801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1303/2592] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:35 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:35.616011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f33768-f8fd-4543-ac55-cbc860adc887 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:35.626411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f33768-f8fd-4543-ac55-cbc860adc887 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:35.632250 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78f0be-4e17-491b-8078-5789ae56894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:35.632771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78f0be-4e17-491b-8078-5789ae56894a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.633053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78f0be-4e17-491b-8078-5789ae56894a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.633353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78f0be-4e17-491b-8078-5789ae56894a admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.634007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78f0be-4e17-491b-8078-5789ae56894a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:35.635670 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78f0be-4e17-491b-8078-5789ae56894a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:35.647362 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1290/2593] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:35.651508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd13c6-87ca-4f4f-afe6-2de7ea648752 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:35.663043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd13c6-87ca-4f4f-afe6-2de7ea648752 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:35.667679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb5954-6568-4cc2-90d8-6e218c65c9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:35.668273 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb5954-6568-4cc2-90d8-6e218c65c9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.669096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb5954-6568-4cc2-90d8-6e218c65c9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.669767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb5954-6568-4cc2-90d8-6e218c65c9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.670497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb5954-6568-4cc2-90d8-6e218c65c9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:35.672103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb5954-6568-4cc2-90d8-6e218c65c9b8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:35.682287 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1304/2594] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:35.687783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78f0be-4e17-491b-8078-5789ae56894a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:35.698183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78f0be-4e17-491b-8078-5789ae56894a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:35.703827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2fc23-17ac-48c9-ab59-f6729bbbb4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:35.704477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2fc23-17ac-48c9-ab59-f6729bbbb4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.704757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2fc23-17ac-48c9-ab59-f6729bbbb4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.705030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2fc23-17ac-48c9-ab59-f6729bbbb4c1 admin admin] PATH_INFO: `/v3/projects/e19997fe7ee748d797eb3351a5ca119c/users/543168a9ae974c37931529e0ac74e3c2/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.721448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2fc23-17ac-48c9-ab59-f6729bbbb4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19997fe7ee748d797eb3351a5ca119c, user_id=543168a9ae974c37931529e0ac74e3c2, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:35.724606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2fc23-17ac-48c9-ab59-f6729bbbb4c1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:35.727398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:35.743660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1291/2595] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:35 2022] PUT /identity/v3/projects/e19997fe7ee748d797eb3351a5ca119c/users/543168a9ae974c37931529e0ac74e3c2/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:35.754852 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547cbd5-7b9d-493a-b918-7886cca1d224 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.755165 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547cbd5-7b9d-493a-b918-7886cca1d224 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.755440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547cbd5-7b9d-493a-b918-7886cca1d224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.798220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7547cbd5-7b9d-493a-b918-7886cca1d224 None None] MFA Rules not processed for user `543168a9ae974c37931529e0ac74e3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:35.799344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7547cbd5-7b9d-493a-b918-7886cca1d224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:35.879301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1305/2596] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:35.891673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2fc23-17ac-48c9-ab59-f6729bbbb4c1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:35.893577 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-31d2fc23-17ac-48c9-ab59-f6729bbbb4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:35.958484 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2fc23-17ac-48c9-ab59-f6729bbbb4c1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:35.963547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa24af-9130-4f3f-b893-ce4046fa4361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:35.964156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa24af-9130-4f3f-b893-ce4046fa4361 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:35.964434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa24af-9130-4f3f-b893-ce4046fa4361 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:35.964708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa24af-9130-4f3f-b893-ce4046fa4361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:35.974505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa24af-9130-4f3f-b893-ce4046fa4361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:35.975928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa24af-9130-4f3f-b893-ce4046fa4361 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:35.989907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1292/2597] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:37.033445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b592c-4ae8-4190-a33c-94290f4a9f0a None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.034075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b592c-4ae8-4190-a33c-94290f4a9f0a None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.034197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b592c-4ae8-4190-a33c-94290f4a9f0a None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.035643 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1306/2598] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:37.042288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e948e-7bb6-48ab-a2f6-d1826f462cbf None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.042777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e948e-7bb6-48ab-a2f6-d1826f462cbf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.042999 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e948e-7bb6-48ab-a2f6-d1826f462cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.064271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ca0e948e-7bb6-48ab-a2f6-d1826f462cbf None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:37.065539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ca0e948e-7bb6-48ab-a2f6-d1826f462cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:37.136103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1293/2599] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:08:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:37.149111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b592c-4ae8-4190-a33c-94290f4a9f0a None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:37.150775 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3a9b592c-4ae8-4190-a33c-94290f4a9f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:37.194999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:37.212464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b592c-4ae8-4190-a33c-94290f4a9f0a None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:37.216721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00168aa2-22e4-46a1-850b-f68669068929 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a47e77cc1942618a3584862042c5f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:37.217386 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168aa2-22e4-46a1-850b-f68669068929 service glance] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.217661 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168aa2-22e4-46a1-850b-f68669068929 service glance] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.217929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168aa2-22e4-46a1-850b-f68669068929 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.227996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00168aa2-22e4-46a1-850b-f68669068929 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:37.229457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00168aa2-22e4-46a1-850b-f68669068929 service glance] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:37.240522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1307/2600] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:37.310186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132d10a-88be-453e-a87f-bb15de2274aa None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.310186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132d10a-88be-453e-a87f-bb15de2274aa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.310515 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132d10a-88be-453e-a87f-bb15de2274aa None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.311871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1294/2601] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:37.316475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8fab2-4942-44ae-89a7-f1ca85a89bda None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.316757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8fab2-4942-44ae-89a7-f1ca85a89bda None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.317031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8fab2-4942-44ae-89a7-f1ca85a89bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.337341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-59a8fab2-4942-44ae-89a7-f1ca85a89bda None None] MFA Rules not processed for user `1a5d841380c243209f55314b545ff56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:37.338536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-59a8fab2-4942-44ae-89a7-f1ca85a89bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:37.397099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1308/2602] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:08:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:37.409019 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132d10a-88be-453e-a87f-bb15de2274aa None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:37.410732 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3132d10a-88be-453e-a87f-bb15de2274aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:37.423584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132d10a-88be-453e-a87f-bb15de2274aa None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:37.427067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e4f91-d85b-4a51-a14b-0b3750cd4496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:37.427644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e4f91-d85b-4a51-a14b-0b3750cd4496 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.427918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e4f91-d85b-4a51-a14b-0b3750cd4496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.428184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e4f91-d85b-4a51-a14b-0b3750cd4496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.435706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e4f91-d85b-4a51-a14b-0b3750cd4496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:37.437101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e4f91-d85b-4a51-a14b-0b3750cd4496 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:37.444765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1295/2603] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:37.529713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f410d-3490-4799-8d8e-757ad8b9b431 None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.529997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f410d-3490-4799-8d8e-757ad8b9b431 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.530286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f410d-3490-4799-8d8e-757ad8b9b431 None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.531878 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1309/2604] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:37.538158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368a8f7-edac-42d5-a6d6-a3ca4b6861ce None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.538560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368a8f7-edac-42d5-a6d6-a3ca4b6861ce None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.539285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368a8f7-edac-42d5-a6d6-a3ca4b6861ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.563863 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1368a8f7-edac-42d5-a6d6-a3ca4b6861ce None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:37.565252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1368a8f7-edac-42d5-a6d6-a3ca4b6861ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:37.576779 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1296/2605] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:08:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:37.587822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f410d-3490-4799-8d8e-757ad8b9b431 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:37.603007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f410d-3490-4799-8d8e-757ad8b9b431 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:37.608133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e645e-a958-4077-903e-ff8fd6e76671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:37.608868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e645e-a958-4077-903e-ff8fd6e76671 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.609299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e645e-a958-4077-903e-ff8fd6e76671 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.609676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e645e-a958-4077-903e-ff8fd6e76671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.625053 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e645e-a958-4077-903e-ff8fd6e76671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:37.626629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e645e-a958-4077-903e-ff8fd6e76671 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:37.643229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1310/2606] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:38.697690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368a8f7-edac-42d5-a6d6-a3ca4b6861ce None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:38.699069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1368a8f7-edac-42d5-a6d6-a3ca4b6861ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:38.766057 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368a8f7-edac-42d5-a6d6-a3ca4b6861ce None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:38.770635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:38.771343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:38.771536 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:38.771807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] PATH_INFO: `/v3/users/5818559dc01341cb9bca172ee6391eef` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:38.772919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5818559dc01341cb9bca172ee6391eef)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:38.774865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:38.785790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:38.820226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5818559dc01341cb9bca172ee6391eef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:38.828290 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5818559dc01341cb9bca172ee6391eef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:38.828984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5818559dc01341cb9bca172ee6391eef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:38.829654 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5818559dc01341cb9bca172ee6391eef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:38.842910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5818559dc01341cb9bca172ee6391eef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:38.854887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1297/2607] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:38 2022] DELETE /identity/v3/users/5818559dc01341cb9bca172ee6391eef => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:38.859752 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914535ca-e12c-4cfc-9848-e286324e61b6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:38.860258 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914535ca-e12c-4cfc-9848-e286324e61b6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:38.860353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914535ca-e12c-4cfc-9848-e286324e61b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:38.880748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-914535ca-e12c-4cfc-9848-e286324e61b6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:38.882068 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-914535ca-e12c-4cfc-9848-e286324e61b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:38.940355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1311/2608] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:38.950938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:38.952618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:39.011087 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918b0a6-ebcb-429d-aa68-97f27cb6cd49 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:39.015564 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59834d3a-45b3-4ac4-8d2d-7ad3df732f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:39.016175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59834d3a-45b3-4ac4-8d2d-7ad3df732f12 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.016460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59834d3a-45b3-4ac4-8d2d-7ad3df732f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.016730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59834d3a-45b3-4ac4-8d2d-7ad3df732f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.026185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59834d3a-45b3-4ac4-8d2d-7ad3df732f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:39.027605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59834d3a-45b3-4ac4-8d2d-7ad3df732f12 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:39.037545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1298/2609] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:39.305711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914535ca-e12c-4cfc-9848-e286324e61b6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:39.307209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-914535ca-e12c-4cfc-9848-e286324e61b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:39.321803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914535ca-e12c-4cfc-9848-e286324e61b6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:39.327435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:39.328076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.328399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.328648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] PATH_INFO: `/v3/projects/d6e9c4e4fe8f453d9c56986296a22571` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.329850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e9c4e4fe8f453d9c56986296a22571)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:39.331813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:39.380893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e9c4e4fe8f453d9c56986296a22571', 'request_id': 'req-b57ee57b-36c0-476e-9a89-b4f5541526e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:39.387518 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e9c4e4fe8f453d9c56986296a22571', 'request_id': 'req-b57ee57b-36c0-476e-9a89-b4f5541526e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:39.395136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e9c4e4fe8f453d9c56986296a22571', 'request_id': 'req-b57ee57b-36c0-476e-9a89-b4f5541526e0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:39.402940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:39.407662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] The token cache is being invalidate because project d6e9c4e4fe8f453d9c56986296a22571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:39.408251 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:39.409578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b57ee57b-36c0-476e-9a89-b4f5541526e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:39.411509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1312/2610] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:39 2022] DELETE /identity/v3/projects/d6e9c4e4fe8f453d9c56986296a22571 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:39.416620 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0693a3a-52a2-4593-b8d1-75d5c418b7b5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.416973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0693a3a-52a2-4593-b8d1-75d5c418b7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.417332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0693a3a-52a2-4593-b8d1-75d5c418b7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.443024 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c0693a3a-52a2-4593-b8d1-75d5c418b7b5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:39.444587 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c0693a3a-52a2-4593-b8d1-75d5c418b7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:39.507021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1299/2611] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:39.514772 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311c0ea-8f58-44bc-a775-0e287edd3849 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.515310 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311c0ea-8f58-44bc-a775-0e287edd3849 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.515658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311c0ea-8f58-44bc-a775-0e287edd3849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.539934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0311c0ea-8f58-44bc-a775-0e287edd3849 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:39.541172 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0311c0ea-8f58-44bc-a775-0e287edd3849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:39.549704 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1313/2612] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:39.554010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0693a3a-52a2-4593-b8d1-75d5c418b7b5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:39.564025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0693a3a-52a2-4593-b8d1-75d5c418b7b5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:39.568626 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8474f8b-5bc8-4a83-adbd-944485771698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:39.569265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8474f8b-5bc8-4a83-adbd-944485771698 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.569546 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8474f8b-5bc8-4a83-adbd-944485771698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.569814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8474f8b-5bc8-4a83-adbd-944485771698 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.570451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8474f8b-5bc8-4a83-adbd-944485771698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:39.570750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8474f8b-5bc8-4a83-adbd-944485771698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:39.572319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8474f8b-5bc8-4a83-adbd-944485771698 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:39.583996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1300/2613] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:39.588154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311c0ea-8f58-44bc-a775-0e287edd3849 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:39.597957 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311c0ea-8f58-44bc-a775-0e287edd3849 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:39.602625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f140a91-0b0c-4474-a575-90597aa0c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:39.603228 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f140a91-0b0c-4474-a575-90597aa0c20c admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.603493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f140a91-0b0c-4474-a575-90597aa0c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.603766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f140a91-0b0c-4474-a575-90597aa0c20c admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.604389 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f140a91-0b0c-4474-a575-90597aa0c20c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:39.607008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f140a91-0b0c-4474-a575-90597aa0c20c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:39.634150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1314/2614] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:39 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:39.638496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8474f8b-5bc8-4a83-adbd-944485771698 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:39.648615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8474f8b-5bc8-4a83-adbd-944485771698 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:39.661343 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:39.705825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e824d-1d41-48ad-81da-d596226de9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:39.706486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e824d-1d41-48ad-81da-d596226de9ca admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.706676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e824d-1d41-48ad-81da-d596226de9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.706944 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e824d-1d41-48ad-81da-d596226de9ca admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.707558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e824d-1d41-48ad-81da-d596226de9ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:39.709479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e824d-1d41-48ad-81da-d596226de9ca admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:39.730269 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1301/2615] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:39 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:39.734946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f140a91-0b0c-4474-a575-90597aa0c20c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:39.745052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f140a91-0b0c-4474-a575-90597aa0c20c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:39.750030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9dc36-0b7f-4f3b-96e8-0e31f0c33c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:39.750619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9dc36-0b7f-4f3b-96e8-0e31f0c33c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.750892 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9dc36-0b7f-4f3b-96e8-0e31f0c33c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.751157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9dc36-0b7f-4f3b-96e8-0e31f0c33c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.751788 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9dc36-0b7f-4f3b-96e8-0e31f0c33c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:39.753394 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9dc36-0b7f-4f3b-96e8-0e31f0c33c03 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:39.763709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1315/2616] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:39.767651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e824d-1d41-48ad-81da-d596226de9ca admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:39.777849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e824d-1d41-48ad-81da-d596226de9ca admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:39.783780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a908e81-ee6d-4858-9c36-09131b4afe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:39.784372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a908e81-ee6d-4858-9c36-09131b4afe8f admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.784648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a908e81-ee6d-4858-9c36-09131b4afe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.784918 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a908e81-ee6d-4858-9c36-09131b4afe8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.785593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a908e81-ee6d-4858-9c36-09131b4afe8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:39.787215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a908e81-ee6d-4858-9c36-09131b4afe8f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:39.801346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1302/2617] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:39.805650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9dc36-0b7f-4f3b-96e8-0e31f0c33c03 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:39.815571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9dc36-0b7f-4f3b-96e8-0e31f0c33c03 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:39.820721 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f778c-4542-4cb9-b9e9-f356206b73ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:39.821416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f778c-4542-4cb9-b9e9-f356206b73ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.821697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f778c-4542-4cb9-b9e9-f356206b73ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.821970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f778c-4542-4cb9-b9e9-f356206b73ea admin admin] PATH_INFO: `/v3/projects/1f795b11281a4739a93ab80ee8b97bf3/users/24f128f1df874bb5ae90ff01d21baa43/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.836571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f778c-4542-4cb9-b9e9-f356206b73ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f795b11281a4739a93ab80ee8b97bf3, user_id=24f128f1df874bb5ae90ff01d21baa43, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:39.839257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f778c-4542-4cb9-b9e9-f356206b73ea admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:39.852627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1316/2618] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:39 2022] PUT /identity/v3/projects/1f795b11281a4739a93ab80ee8b97bf3/users/24f128f1df874bb5ae90ff01d21baa43/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:39.863583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ee636-dfc3-4971-b2d7-b7edf1fdd4b2 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:39.863872 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ee636-dfc3-4971-b2d7-b7edf1fdd4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:39.864149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ee636-dfc3-4971-b2d7-b7edf1fdd4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:39.892644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-547ee636-dfc3-4971-b2d7-b7edf1fdd4b2 None None] MFA Rules not processed for user `24f128f1df874bb5ae90ff01d21baa43`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:39.893923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-547ee636-dfc3-4971-b2d7-b7edf1fdd4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:39.957051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1303/2619] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:39.961654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f778c-4542-4cb9-b9e9-f356206b73ea admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.022638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f778c-4542-4cb9-b9e9-f356206b73ea admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.027158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e568b6-aad5-4fdc-a83c-177ed013a714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.027753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e568b6-aad5-4fdc-a83c-177ed013a714 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.028698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e568b6-aad5-4fdc-a83c-177ed013a714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.029323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e568b6-aad5-4fdc-a83c-177ed013a714 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.029954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e568b6-aad5-4fdc-a83c-177ed013a714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.031869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e568b6-aad5-4fdc-a83c-177ed013a714 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.057685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1317/2620] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:39 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:40.062231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ee636-dfc3-4971-b2d7-b7edf1fdd4b2 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.072750 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ee636-dfc3-4971-b2d7-b7edf1fdd4b2 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.127383 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c49b4c-3eb4-4e06-8a0c-c16b2b81b5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.127964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c49b4c-3eb4-4e06-8a0c-c16b2b81b5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.128237 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c49b4c-3eb4-4e06-8a0c-c16b2b81b5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.128504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c49b4c-3eb4-4e06-8a0c-c16b2b81b5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.129150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c49b4c-3eb4-4e06-8a0c-c16b2b81b5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.131027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c49b4c-3eb4-4e06-8a0c-c16b2b81b5e0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.152517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1304/2621] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:40 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:40.157514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e568b6-aad5-4fdc-a83c-177ed013a714 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.167677 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e568b6-aad5-4fdc-a83c-177ed013a714 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.172235 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67954c-5b20-40b3-b5f8-a7943e08a9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.172827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67954c-5b20-40b3-b5f8-a7943e08a9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.173141 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67954c-5b20-40b3-b5f8-a7943e08a9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.173415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67954c-5b20-40b3-b5f8-a7943e08a9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.174047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67954c-5b20-40b3-b5f8-a7943e08a9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.175688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67954c-5b20-40b3-b5f8-a7943e08a9c8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.187076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1318/2622] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:40.190926 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c49b4c-3eb4-4e06-8a0c-c16b2b81b5e0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.201345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c49b4c-3eb4-4e06-8a0c-c16b2b81b5e0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.205903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cee619-800d-496c-a83b-7f123ac335bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.206482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cee619-800d-496c-a83b-7f123ac335bd admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.206758 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cee619-800d-496c-a83b-7f123ac335bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.207026 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cee619-800d-496c-a83b-7f123ac335bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.207656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cee619-800d-496c-a83b-7f123ac335bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.209244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cee619-800d-496c-a83b-7f123ac335bd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.220189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1305/2623] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:40.224149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67954c-5b20-40b3-b5f8-a7943e08a9c8 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.234249 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67954c-5b20-40b3-b5f8-a7943e08a9c8 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.238713 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b810d7-1f01-44bf-a049-65c8d9dbf5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.239344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b810d7-1f01-44bf-a049-65c8d9dbf5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.239617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b810d7-1f01-44bf-a049-65c8d9dbf5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.239881 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b810d7-1f01-44bf-a049-65c8d9dbf5d1 admin admin] PATH_INFO: `/v3/projects/175a2a5769244dc9b06f9cec08974eb5/users/ebb57c163de2407aaa2a8d629e76cd1b/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.254344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b810d7-1f01-44bf-a049-65c8d9dbf5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175a2a5769244dc9b06f9cec08974eb5, user_id=ebb57c163de2407aaa2a8d629e76cd1b, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.259363 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b810d7-1f01-44bf-a049-65c8d9dbf5d1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.272157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1319/2624] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:40 2022] PUT /identity/v3/projects/175a2a5769244dc9b06f9cec08974eb5/users/ebb57c163de2407aaa2a8d629e76cd1b/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:40.284465 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd51b5-27df-487f-a220-9b1da285b77f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.284720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd51b5-27df-487f-a220-9b1da285b77f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.285035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd51b5-27df-487f-a220-9b1da285b77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.293060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:40.315830 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6cbd51b5-27df-487f-a220-9b1da285b77f None None] MFA Rules not processed for user `ebb57c163de2407aaa2a8d629e76cd1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:40.317041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6cbd51b5-27df-487f-a220-9b1da285b77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:40.383361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1306/2625] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:40.388445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b810d7-1f01-44bf-a049-65c8d9dbf5d1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.435401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:40.451269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b810d7-1f01-44bf-a049-65c8d9dbf5d1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.455950 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea99c7-c0aa-46be-985d-f5a087c5059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.456547 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea99c7-c0aa-46be-985d-f5a087c5059b admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.456827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea99c7-c0aa-46be-985d-f5a087c5059b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.457122 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea99c7-c0aa-46be-985d-f5a087c5059b admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.457755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea99c7-c0aa-46be-985d-f5a087c5059b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.459659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea99c7-c0aa-46be-985d-f5a087c5059b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.485351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1320/2626] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:40 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:40.489522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd51b5-27df-487f-a220-9b1da285b77f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.500895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd51b5-27df-487f-a220-9b1da285b77f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.560364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91218e-04d2-4a29-a7cf-a3e4b0e424ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.561303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91218e-04d2-4a29-a7cf-a3e4b0e424ec admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.561425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91218e-04d2-4a29-a7cf-a3e4b0e424ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.561751 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91218e-04d2-4a29-a7cf-a3e4b0e424ec admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.562316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91218e-04d2-4a29-a7cf-a3e4b0e424ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.564333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91218e-04d2-4a29-a7cf-a3e4b0e424ec admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.601501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1307/2627] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:40 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:40.605982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea99c7-c0aa-46be-985d-f5a087c5059b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.620564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea99c7-c0aa-46be-985d-f5a087c5059b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.627740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecdc712-29c0-4b3f-91a1-6a8b1db786af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.627740 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecdc712-29c0-4b3f-91a1-6a8b1db786af admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.628312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecdc712-29c0-4b3f-91a1-6a8b1db786af admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.628607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecdc712-29c0-4b3f-91a1-6a8b1db786af admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.629797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecdc712-29c0-4b3f-91a1-6a8b1db786af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.631328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecdc712-29c0-4b3f-91a1-6a8b1db786af admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.643651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1321/2628] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:40.647846 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91218e-04d2-4a29-a7cf-a3e4b0e424ec admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.662616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91218e-04d2-4a29-a7cf-a3e4b0e424ec admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.670678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74632d7-3545-44be-a81f-d9c944588b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.672816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74632d7-3545-44be-a81f-d9c944588b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.673717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74632d7-3545-44be-a81f-d9c944588b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.675186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74632d7-3545-44be-a81f-d9c944588b7f admin admin] PATH_INFO: `/v3/projects/2f22c0b132dd4d8eb915d5631cced8d5/users/9ca5a3783aa94988bbf893c5cd017b46/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.695929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74632d7-3545-44be-a81f-d9c944588b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f22c0b132dd4d8eb915d5631cced8d5, user_id=9ca5a3783aa94988bbf893c5cd017b46, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.698777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74632d7-3545-44be-a81f-d9c944588b7f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.714425 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1308/2629] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:40 2022] PUT /identity/v3/projects/2f22c0b132dd4d8eb915d5631cced8d5/users/9ca5a3783aa94988bbf893c5cd017b46/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:40.718900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecdc712-29c0-4b3f-91a1-6a8b1db786af admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.729786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecdc712-29c0-4b3f-91a1-6a8b1db786af admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.797731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550541e3-f83b-4d53-9388-1bb16b0e2bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.798404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550541e3-f83b-4d53-9388-1bb16b0e2bdc admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.798690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550541e3-f83b-4d53-9388-1bb16b0e2bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.798959 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550541e3-f83b-4d53-9388-1bb16b0e2bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.799604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550541e3-f83b-4d53-9388-1bb16b0e2bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.801245 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550541e3-f83b-4d53-9388-1bb16b0e2bdc admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.814625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1322/2630] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:40.818622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74632d7-3545-44be-a81f-d9c944588b7f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.831078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74632d7-3545-44be-a81f-d9c944588b7f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:40.836203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e80ef-bb49-41a5-afcd-fe7bcc698ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:40.836860 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e80ef-bb49-41a5-afcd-fe7bcc698ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.837164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e80ef-bb49-41a5-afcd-fe7bcc698ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.837440 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e80ef-bb49-41a5-afcd-fe7bcc698ee2 admin admin] PATH_INFO: `/v3/projects/2f22c0b132dd4d8eb915d5631cced8d5/users/9ca5a3783aa94988bbf893c5cd017b46/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.839372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e80ef-bb49-41a5-afcd-fe7bcc698ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f22c0b132dd4d8eb915d5631cced8d5, user_id=9ca5a3783aa94988bbf893c5cd017b46, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:40.842073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e80ef-bb49-41a5-afcd-fe7bcc698ee2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:40.852790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1309/2631] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:40 2022] PUT /identity/v3/projects/2f22c0b132dd4d8eb915d5631cced8d5/users/9ca5a3783aa94988bbf893c5cd017b46/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:40.863668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828951c-858a-44d2-bb07-babca2f58939 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:40.863955 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828951c-858a-44d2-bb07-babca2f58939 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:40.864298 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828951c-858a-44d2-bb07-babca2f58939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:40.894488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d828951c-858a-44d2-bb07-babca2f58939 None None] MFA Rules not processed for user `9ca5a3783aa94988bbf893c5cd017b46`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:40.895749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d828951c-858a-44d2-bb07-babca2f58939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:40.973682 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1323/2632] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:40.992847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e80ef-bb49-41a5-afcd-fe7bcc698ee2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:40.996448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-857e80ef-bb49-41a5-afcd-fe7bcc698ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:41.056096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:41.061716 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e80ef-bb49-41a5-afcd-fe7bcc698ee2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:41.066394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddf871-74d1-4221-be2a-5d2de2959d72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:41.067507 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddf871-74d1-4221-be2a-5d2de2959d72 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:41.067934 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddf871-74d1-4221-be2a-5d2de2959d72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:41.068213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddf871-74d1-4221-be2a-5d2de2959d72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:41.129772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddf871-74d1-4221-be2a-5d2de2959d72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:41.131204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddf871-74d1-4221-be2a-5d2de2959d72 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:41.140701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1310/2633] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:44.772401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828951c-858a-44d2-bb07-babca2f58939 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:44.784012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828951c-858a-44d2-bb07-babca2f58939 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:44.787549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327163d2-a3ee-47a0-9ecd-1639a2f60d8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:44.788143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327163d2-a3ee-47a0-9ecd-1639a2f60d8b service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:44.788419 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327163d2-a3ee-47a0-9ecd-1639a2f60d8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:44.788685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327163d2-a3ee-47a0-9ecd-1639a2f60d8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:44.834075 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:44.851839 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327163d2-a3ee-47a0-9ecd-1639a2f60d8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:44.853357 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327163d2-a3ee-47a0-9ecd-1639a2f60d8b service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:44.862954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1324/2634] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:44.942559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a930b4-5cc3-4eaa-9aca-585110f2143f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:44.943169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a930b4-5cc3-4eaa-9aca-585110f2143f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:44.943169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a930b4-5cc3-4eaa-9aca-585110f2143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:44.963496 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a6a930b4-5cc3-4eaa-9aca-585110f2143f None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:44.965018 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a6a930b4-5cc3-4eaa-9aca-585110f2143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:45.023469 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1311/2635] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:08:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:45.036733 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327163d2-a3ee-47a0-9ecd-1639a2f60d8b service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:45.038576 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-327163d2-a3ee-47a0-9ecd-1639a2f60d8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:45.112231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327163d2-a3ee-47a0-9ecd-1639a2f60d8b service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:45.117792 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5735d23-7dfe-4294-b60b-c8d389c12510 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:45.118502 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5735d23-7dfe-4294-b60b-c8d389c12510 service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:45.118820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5735d23-7dfe-4294-b60b-c8d389c12510 service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:45.119127 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5735d23-7dfe-4294-b60b-c8d389c12510 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:45.131215 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5735d23-7dfe-4294-b60b-c8d389c12510 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:45.132910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5735d23-7dfe-4294-b60b-c8d389c12510 service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:45.144050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1325/2636] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:08:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:08:45.715140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a930b4-5cc3-4eaa-9aca-585110f2143f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:45.729365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a930b4-5cc3-4eaa-9aca-585110f2143f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:45.737789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6baac65-a34a-4bb0-9b56-7fdeb93b2186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:45.738442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baac65-a34a-4bb0-9b56-7fdeb93b2186 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:45.738720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baac65-a34a-4bb0-9b56-7fdeb93b2186 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:45.738992 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baac65-a34a-4bb0-9b56-7fdeb93b2186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:45.762982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6baac65-a34a-4bb0-9b56-7fdeb93b2186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:45.764887 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6baac65-a34a-4bb0-9b56-7fdeb93b2186 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:45.787084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1312/2637] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:53.253315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5735d23-7dfe-4294-b60b-c8d389c12510 service swift] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:53.321685 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5735d23-7dfe-4294-b60b-c8d389c12510 service swift] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:53.327259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:53.327883 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:53.328164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:53.328434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] PATH_INFO: `/v3/users/24f128f1df874bb5ae90ff01d21baa43` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:53.329659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f128f1df874bb5ae90ff01d21baa43)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:53.331585 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:53.379027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f128f1df874bb5ae90ff01d21baa43'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:53.386402 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f128f1df874bb5ae90ff01d21baa43'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:53.394230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f128f1df874bb5ae90ff01d21baa43'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:53.394871 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f128f1df874bb5ae90ff01d21baa43'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:53.401744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:53.409563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f128f1df874bb5ae90ff01d21baa43'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:53.415625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1326/2638] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:53 2022] DELETE /identity/v3/users/24f128f1df874bb5ae90ff01d21baa43 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:53.420721 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc03c5b-d297-45e9-b61a-0cff6e80b517 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:53.421329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc03c5b-d297-45e9-b61a-0cff6e80b517 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:53.421329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc03c5b-d297-45e9-b61a-0cff6e80b517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:53.441567 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7fc03c5b-d297-45e9-b61a-0cff6e80b517 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:53.442822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7fc03c5b-d297-45e9-b61a-0cff6e80b517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:53.508331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1313/2639] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:53.518343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:53.519802 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:53.578457 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e01ec-3c9c-4f91-90d2-1234110ee10b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:53.582254 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d89a6-604d-4a77-a1cf-33a56f97416f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:53.582889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d89a6-604d-4a77-a1cf-33a56f97416f service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:53.583171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d89a6-604d-4a77-a1cf-33a56f97416f service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:53.583508 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d89a6-604d-4a77-a1cf-33a56f97416f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:53.592980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d89a6-604d-4a77-a1cf-33a56f97416f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:53.594497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d89a6-604d-4a77-a1cf-33a56f97416f service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:53.604644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1327/2640] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:53.876679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc03c5b-d297-45e9-b61a-0cff6e80b517 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:53.878715 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7fc03c5b-d297-45e9-b61a-0cff6e80b517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:53.908890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc03c5b-d297-45e9-b61a-0cff6e80b517 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:53.916842 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:53.917598 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:53.917836 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:53.918108 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] PATH_INFO: `/v3/projects/1f795b11281a4739a93ab80ee8b97bf3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:53.919702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f795b11281a4739a93ab80ee8b97bf3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:53.922396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:53.971208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:53.986084 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f795b11281a4739a93ab80ee8b97bf3', 'request_id': 'req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:53.992349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f795b11281a4739a93ab80ee8b97bf3', 'request_id': 'req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.000788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f795b11281a4739a93ab80ee8b97bf3', 'request_id': 'req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.011160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] The token cache is being invalidate because project 1f795b11281a4739a93ab80ee8b97bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:54.011466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.012073 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.014274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1314/2641] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:53 2022] DELETE /identity/v3/projects/1f795b11281a4739a93ab80ee8b97bf3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:54.018270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d89a6-604d-4a77-a1cf-33a56f97416f service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:54.027889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d89a6-604d-4a77-a1cf-33a56f97416f service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:54.082956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:54.083600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:54.083876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:54.084143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] PATH_INFO: `/v3/users/ebb57c163de2407aaa2a8d629e76cd1b` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:54.085316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb57c163de2407aaa2a8d629e76cd1b)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:54.087279 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:54.134641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb57c163de2407aaa2a8d629e76cd1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.141730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb57c163de2407aaa2a8d629e76cd1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.148541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb57c163de2407aaa2a8d629e76cd1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.149200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb57c163de2407aaa2a8d629e76cd1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.161491 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb57c163de2407aaa2a8d629e76cd1b'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.167229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1328/2642] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:54 2022] DELETE /identity/v3/users/ebb57c163de2407aaa2a8d629e76cd1b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:54.486035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:54.487907 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:54.511668 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82fdc7-fd15-4a31-a934-ac7d344e8c4d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:54.581653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:54.582275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:54.582552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:54.582823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] PATH_INFO: `/v3/projects/175a2a5769244dc9b06f9cec08974eb5` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:54.584000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] RBAC: Authorizing `identity:delete_project(project_id=175a2a5769244dc9b06f9cec08974eb5)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:54.585914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:54.631051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175a2a5769244dc9b06f9cec08974eb5', 'request_id': 'req-5bbebd9e-c01c-403d-8cec-658fc6da828d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.636958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175a2a5769244dc9b06f9cec08974eb5', 'request_id': 'req-5bbebd9e-c01c-403d-8cec-658fc6da828d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.644890 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175a2a5769244dc9b06f9cec08974eb5', 'request_id': 'req-5bbebd9e-c01c-403d-8cec-658fc6da828d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.655909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] The token cache is being invalidate because project 175a2a5769244dc9b06f9cec08974eb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:54.656199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.656817 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.658309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1315/2643] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:54 2022] DELETE /identity/v3/projects/175a2a5769244dc9b06f9cec08974eb5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:54.662446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:54.663870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:54.676473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1110fbb-b097-442a-b640-45bc015b6804 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:54.683833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:54.734756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:54.735296 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:54.735578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:54.735850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] PATH_INFO: `/v3/users/9ca5a3783aa94988bbf893c5cd017b46` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:54.737036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca5a3783aa94988bbf893c5cd017b46)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:54.739271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:54.793395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca5a3783aa94988bbf893c5cd017b46'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.801362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca5a3783aa94988bbf893c5cd017b46'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.808205 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca5a3783aa94988bbf893c5cd017b46'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.808840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca5a3783aa94988bbf893c5cd017b46'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.821013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-0ca25f6b-3585-49d7-8970-d75063ec8c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca5a3783aa94988bbf893c5cd017b46'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:54.827843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1329/2644] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:54 2022] DELETE /identity/v3/users/9ca5a3783aa94988bbf893c5cd017b46 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:55.096833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:55.098352 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:55.112991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbebd9e-c01c-403d-8cec-658fc6da828d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:55.159454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:55.173864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:55.174489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.174767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.175041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] PATH_INFO: `/v3/projects/2f22c0b132dd4d8eb915d5631cced8d5` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.176140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f22c0b132dd4d8eb915d5631cced8d5)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:55.178184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:55.225294 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f22c0b132dd4d8eb915d5631cced8d5', 'request_id': 'req-1ba0751c-a490-4e3a-a5e5-54ccbf607052'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:55.231318 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f22c0b132dd4d8eb915d5631cced8d5', 'request_id': 'req-1ba0751c-a490-4e3a-a5e5-54ccbf607052'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:55.238618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f22c0b132dd4d8eb915d5631cced8d5', 'request_id': 'req-1ba0751c-a490-4e3a-a5e5-54ccbf607052'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:55.250260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] The token cache is being invalidate because project 2f22c0b132dd4d8eb915d5631cced8d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:55.250547 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:55.251185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1ba0751c-a490-4e3a-a5e5-54ccbf607052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:55.252657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1316/2645] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:55 2022] DELETE /identity/v3/projects/2f22c0b132dd4d8eb915d5631cced8d5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:55.257588 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c646fb-c337-4831-bd7f-cab0bf01c849 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.257989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c646fb-c337-4831-bd7f-cab0bf01c849 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.258387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c646fb-c337-4831-bd7f-cab0bf01c849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.279610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-11c646fb-c337-4831-bd7f-cab0bf01c849 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:55.280822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-11c646fb-c337-4831-bd7f-cab0bf01c849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:55.340361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1330/2646] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:55.348400 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d02f65-cfad-420e-b4e1-cdc3b4168822 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.348684 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d02f65-cfad-420e-b4e1-cdc3b4168822 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.348961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d02f65-cfad-420e-b4e1-cdc3b4168822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.368413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f5d02f65-cfad-420e-b4e1-cdc3b4168822 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:55.369642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f5d02f65-cfad-420e-b4e1-cdc3b4168822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:55.378719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1317/2647] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:55.382996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c646fb-c337-4831-bd7f-cab0bf01c849 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:55.393442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c646fb-c337-4831-bd7f-cab0bf01c849 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:55.398023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89a8e6-119b-40bc-bcaa-6344a21a06e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:55.398618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89a8e6-119b-40bc-bcaa-6344a21a06e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.398902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89a8e6-119b-40bc-bcaa-6344a21a06e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.399173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89a8e6-119b-40bc-bcaa-6344a21a06e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.399812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89a8e6-119b-40bc-bcaa-6344a21a06e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:55.400113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89a8e6-119b-40bc-bcaa-6344a21a06e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:55.401751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89a8e6-119b-40bc-bcaa-6344a21a06e0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:55.413554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1331/2648] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:55.417664 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d02f65-cfad-420e-b4e1-cdc3b4168822 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:55.426909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d02f65-cfad-420e-b4e1-cdc3b4168822 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:55.431526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db47fd-a2eb-4094-a687-06963fd795ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:55.432111 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db47fd-a2eb-4094-a687-06963fd795ca admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.432382 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db47fd-a2eb-4094-a687-06963fd795ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.432648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db47fd-a2eb-4094-a687-06963fd795ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.433289 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db47fd-a2eb-4094-a687-06963fd795ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:55.435736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db47fd-a2eb-4094-a687-06963fd795ca admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:55.464950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1318/2649] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:55 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:55.469192 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89a8e6-119b-40bc-bcaa-6344a21a06e0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:55.479405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89a8e6-119b-40bc-bcaa-6344a21a06e0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:55.492397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:55.542184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bbb5d-2fa7-47b0-911f-67fd89a8895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:55.542825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bbb5d-2fa7-47b0-911f-67fd89a8895a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.543103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bbb5d-2fa7-47b0-911f-67fd89a8895a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.543375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bbb5d-2fa7-47b0-911f-67fd89a8895a admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.544034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bbb5d-2fa7-47b0-911f-67fd89a8895a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:55.546191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bbb5d-2fa7-47b0-911f-67fd89a8895a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:55.573109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1332/2650] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:55 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:55.576671 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db47fd-a2eb-4094-a687-06963fd795ca admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:55.592319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db47fd-a2eb-4094-a687-06963fd795ca admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:55.597620 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30679bc-9a2b-49cd-b1ba-800b5cb03e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:55.601264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30679bc-9a2b-49cd-b1ba-800b5cb03e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.601264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30679bc-9a2b-49cd-b1ba-800b5cb03e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.601264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30679bc-9a2b-49cd-b1ba-800b5cb03e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.601264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30679bc-9a2b-49cd-b1ba-800b5cb03e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:55.601641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30679bc-9a2b-49cd-b1ba-800b5cb03e74 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:55.614889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1319/2651] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:08:55.618908 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bbb5d-2fa7-47b0-911f-67fd89a8895a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:55.629477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bbb5d-2fa7-47b0-911f-67fd89a8895a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:55.633996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156d8cf-b8f2-4656-a529-2778803f6ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:55.634647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156d8cf-b8f2-4656-a529-2778803f6ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.634923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156d8cf-b8f2-4656-a529-2778803f6ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.635190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156d8cf-b8f2-4656-a529-2778803f6ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.635825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156d8cf-b8f2-4656-a529-2778803f6ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:55.637435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156d8cf-b8f2-4656-a529-2778803f6ea1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:55.648019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1333/2652] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:55.653994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30679bc-9a2b-49cd-b1ba-800b5cb03e74 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:55.664307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30679bc-9a2b-49cd-b1ba-800b5cb03e74 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:55.668968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb273a-1730-4cad-bf24-09d9f4fdd7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:55.670159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb273a-1730-4cad-bf24-09d9f4fdd7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.670442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb273a-1730-4cad-bf24-09d9f4fdd7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.670719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb273a-1730-4cad-bf24-09d9f4fdd7a8 admin admin] PATH_INFO: `/v3/projects/7e68d4489efc47ea9aae9ee7d73ca5ca/users/55c3ede88baa4b8e95e7d4411c5cc6e3/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.686748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb273a-1730-4cad-bf24-09d9f4fdd7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e68d4489efc47ea9aae9ee7d73ca5ca, user_id=55c3ede88baa4b8e95e7d4411c5cc6e3, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:55.689419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb273a-1730-4cad-bf24-09d9f4fdd7a8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:55.703001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1320/2653] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:55 2022] PUT /identity/v3/projects/7e68d4489efc47ea9aae9ee7d73ca5ca/users/55c3ede88baa4b8e95e7d4411c5cc6e3/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:55.713099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2ac97-e374-459c-85d6-cafe622926c7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.713571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2ac97-e374-459c-85d6-cafe622926c7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.713688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2ac97-e374-459c-85d6-cafe622926c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.741369 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-54d2ac97-e374-459c-85d6-cafe622926c7 None None] MFA Rules not processed for user `55c3ede88baa4b8e95e7d4411c5cc6e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:55.742633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-54d2ac97-e374-459c-85d6-cafe622926c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:55.808109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1334/2654] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:55.813705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb273a-1730-4cad-bf24-09d9f4fdd7a8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.001303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb273a-1730-4cad-bf24-09d9f4fdd7a8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.006032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43e115-5aab-4e77-9452-f35c544c2bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.006635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43e115-5aab-4e77-9452-f35c544c2bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.006916 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43e115-5aab-4e77-9452-f35c544c2bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.007183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43e115-5aab-4e77-9452-f35c544c2bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.007815 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43e115-5aab-4e77-9452-f35c544c2bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.009727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43e115-5aab-4e77-9452-f35c544c2bbc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.031065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:56.039505 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1321/2655] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:55 2022] POST /identity/v3/projects => generated 383 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:56.043424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2ac97-e374-459c-85d6-cafe622926c7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.053981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2ac97-e374-459c-85d6-cafe622926c7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.109698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496a452-60b9-463e-b81c-a3275668a2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.110300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496a452-60b9-463e-b81c-a3275668a2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.110573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496a452-60b9-463e-b81c-a3275668a2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.110837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496a452-60b9-463e-b81c-a3275668a2b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.111454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496a452-60b9-463e-b81c-a3275668a2b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.113390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496a452-60b9-463e-b81c-a3275668a2b8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.135557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1335/2656] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:56 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:56.140301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43e115-5aab-4e77-9452-f35c544c2bbc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.150703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43e115-5aab-4e77-9452-f35c544c2bbc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.155928 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b5f96-a3c9-46c6-85b3-9a762bc51df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.156520 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b5f96-a3c9-46c6-85b3-9a762bc51df5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.156796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b5f96-a3c9-46c6-85b3-9a762bc51df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.157065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b5f96-a3c9-46c6-85b3-9a762bc51df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.157731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b5f96-a3c9-46c6-85b3-9a762bc51df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.159304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b5f96-a3c9-46c6-85b3-9a762bc51df5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.169814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1322/2657] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:56.173667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496a452-60b9-463e-b81c-a3275668a2b8 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.183367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496a452-60b9-463e-b81c-a3275668a2b8 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.188513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d65b0f-c078-45c3-9dc5-568f32c5c186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.189110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d65b0f-c078-45c3-9dc5-568f32c5c186 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.189390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d65b0f-c078-45c3-9dc5-568f32c5c186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.189657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d65b0f-c078-45c3-9dc5-568f32c5c186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.190282 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d65b0f-c078-45c3-9dc5-568f32c5c186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.191875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d65b0f-c078-45c3-9dc5-568f32c5c186 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.202133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1336/2658] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:56.206147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b5f96-a3c9-46c6-85b3-9a762bc51df5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.215903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b5f96-a3c9-46c6-85b3-9a762bc51df5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.220472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43710d-7f8c-464a-b4e4-a22bbcff47a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.221130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43710d-7f8c-464a-b4e4-a22bbcff47a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.221407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43710d-7f8c-464a-b4e4-a22bbcff47a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.221673 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43710d-7f8c-464a-b4e4-a22bbcff47a4 admin admin] PATH_INFO: `/v3/projects/0b551f16f8f44b6981be3b7a9cc3410d/users/54c50f3163cb4c44a7b9ed04c2a84ca7/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.235782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43710d-7f8c-464a-b4e4-a22bbcff47a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b551f16f8f44b6981be3b7a9cc3410d, user_id=54c50f3163cb4c44a7b9ed04c2a84ca7, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.238429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43710d-7f8c-464a-b4e4-a22bbcff47a4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.252224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1323/2659] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:56 2022] PUT /identity/v3/projects/0b551f16f8f44b6981be3b7a9cc3410d/users/54c50f3163cb4c44a7b9ed04c2a84ca7/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:56.262566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6e5fa-641e-4dce-b8fc-5396d253602e None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.263562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6e5fa-641e-4dce-b8fc-5396d253602e None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.263875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6e5fa-641e-4dce-b8fc-5396d253602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.271930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:56.295182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-b2c6e5fa-641e-4dce-b8fc-5396d253602e None None] MFA Rules not processed for user `54c50f3163cb4c44a7b9ed04c2a84ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:56.296401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b2c6e5fa-641e-4dce-b8fc-5396d253602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:56.361624 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1337/2660] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:56.366213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43710d-7f8c-464a-b4e4-a22bbcff47a4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.429320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43710d-7f8c-464a-b4e4-a22bbcff47a4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.433880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c697931-a67b-4da5-8725-cced67b65e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.434488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c697931-a67b-4da5-8725-cced67b65e13 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.435417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c697931-a67b-4da5-8725-cced67b65e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.435777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c697931-a67b-4da5-8725-cced67b65e13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.436550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c697931-a67b-4da5-8725-cced67b65e13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.438424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c697931-a67b-4da5-8725-cced67b65e13 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.465161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1324/2661] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:56 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:56.469401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6e5fa-641e-4dce-b8fc-5396d253602e None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.479887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6e5fa-641e-4dce-b8fc-5396d253602e None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.538568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024775d4-3a21-4f99-bcdd-00e476c31145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.539395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024775d4-3a21-4f99-bcdd-00e476c31145 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.539674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024775d4-3a21-4f99-bcdd-00e476c31145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.539942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024775d4-3a21-4f99-bcdd-00e476c31145 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.540564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024775d4-3a21-4f99-bcdd-00e476c31145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.543200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024775d4-3a21-4f99-bcdd-00e476c31145 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.564944 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1338/2662] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:56 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:56.568901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c697931-a67b-4da5-8725-cced67b65e13 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.579508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c697931-a67b-4da5-8725-cced67b65e13 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.584067 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852d777-f90f-4a81-a51e-60287fdb2595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.584679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852d777-f90f-4a81-a51e-60287fdb2595 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.584952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852d777-f90f-4a81-a51e-60287fdb2595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.585254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852d777-f90f-4a81-a51e-60287fdb2595 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.585895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852d777-f90f-4a81-a51e-60287fdb2595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.587478 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852d777-f90f-4a81-a51e-60287fdb2595 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.600954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1325/2663] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:56.601972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024775d4-3a21-4f99-bcdd-00e476c31145 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.613371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024775d4-3a21-4f99-bcdd-00e476c31145 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.618983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d07335-b5e0-46c8-8167-a945eadd75e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.619568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d07335-b5e0-46c8-8167-a945eadd75e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.619880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d07335-b5e0-46c8-8167-a945eadd75e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.620107 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d07335-b5e0-46c8-8167-a945eadd75e9 admin admin] PATH_INFO: `/v3/projects/ff65a6aad874428d8f3e1d4360104254/users/4a155ef4562e460aa05d371d85aad746/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.636903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d07335-b5e0-46c8-8167-a945eadd75e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff65a6aad874428d8f3e1d4360104254, user_id=4a155ef4562e460aa05d371d85aad746, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.639970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d07335-b5e0-46c8-8167-a945eadd75e9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.656919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1339/2664] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:56 2022] PUT /identity/v3/projects/ff65a6aad874428d8f3e1d4360104254/users/4a155ef4562e460aa05d371d85aad746/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:56.661763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852d777-f90f-4a81-a51e-60287fdb2595 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.672823 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852d777-f90f-4a81-a51e-60287fdb2595 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.731307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad79551-a19c-4763-8617-19bb88084d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.732847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79551-a19c-4763-8617-19bb88084d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.733167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79551-a19c-4763-8617-19bb88084d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.733441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79551-a19c-4763-8617-19bb88084d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.734080 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad79551-a19c-4763-8617-19bb88084d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.735641 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad79551-a19c-4763-8617-19bb88084d16 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.745991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1326/2665] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:56.750264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d07335-b5e0-46c8-8167-a945eadd75e9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.760512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d07335-b5e0-46c8-8167-a945eadd75e9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:56.765255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6e80f-951a-44d3-ab97-a1a4b0ea5f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:56.765929 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e80f-951a-44d3-ab97-a1a4b0ea5f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.766159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e80f-951a-44d3-ab97-a1a4b0ea5f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.766463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e80f-951a-44d3-ab97-a1a4b0ea5f70 admin admin] PATH_INFO: `/v3/projects/ff65a6aad874428d8f3e1d4360104254/users/4a155ef4562e460aa05d371d85aad746/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.770343 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6e80f-951a-44d3-ab97-a1a4b0ea5f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff65a6aad874428d8f3e1d4360104254, user_id=4a155ef4562e460aa05d371d85aad746, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:56.772977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6e80f-951a-44d3-ab97-a1a4b0ea5f70 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:56.784167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1340/2666] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:56 2022] PUT /identity/v3/projects/ff65a6aad874428d8f3e1d4360104254/users/4a155ef4562e460aa05d371d85aad746/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:56.852390 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088dea-9afc-4b48-b5d8-70202258546a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:56.852590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088dea-9afc-4b48-b5d8-70202258546a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:56.852847 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088dea-9afc-4b48-b5d8-70202258546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.881698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-09088dea-9afc-4b48-b5d8-70202258546a None None] MFA Rules not processed for user `4a155ef4562e460aa05d371d85aad746`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:56.882893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-09088dea-9afc-4b48-b5d8-70202258546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:56.902942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:56.956781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1327/2667] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:56.968824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6e80f-951a-44d3-ab97-a1a4b0ea5f70 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:56.970589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-60c6e80f-951a-44d3-ab97-a1a4b0ea5f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:57.031148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:57.039923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6e80f-951a-44d3-ab97-a1a4b0ea5f70 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:57.047026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed96a42-52ae-4ce0-9585-deb23f702004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:57.047681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed96a42-52ae-4ce0-9585-deb23f702004 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:57.048447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed96a42-52ae-4ce0-9585-deb23f702004 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:57.048975 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed96a42-52ae-4ce0-9585-deb23f702004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:57.117860 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed96a42-52ae-4ce0-9585-deb23f702004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:57.119291 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed96a42-52ae-4ce0-9585-deb23f702004 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:57.129429 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1341/2668] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:57.801027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09088dea-9afc-4b48-b5d8-70202258546a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:57.802481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-09088dea-9afc-4b48-b5d8-70202258546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:57.869509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09088dea-9afc-4b48-b5d8-70202258546a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:57.874834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:57.875463 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:57.875768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:57.876037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] PATH_INFO: `/v3/users/543168a9ae974c37931529e0ac74e3c2` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:57.877212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=543168a9ae974c37931529e0ac74e3c2)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:57.879163 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:57.930606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '543168a9ae974c37931529e0ac74e3c2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:57.939725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '543168a9ae974c37931529e0ac74e3c2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:57.940424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '543168a9ae974c37931529e0ac74e3c2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:57.941127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '543168a9ae974c37931529e0ac74e3c2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:57.955116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '543168a9ae974c37931529e0ac74e3c2'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:57.968031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1328/2669] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:57 2022] DELETE /identity/v3/users/543168a9ae974c37931529e0ac74e3c2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:57.972496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dfd46-6f3d-420a-8af0-869cc7515c16 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:57.973358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dfd46-6f3d-420a-8af0-869cc7515c16 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:57.973653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dfd46-6f3d-420a-8af0-869cc7515c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:57.992203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8b0dfd46-6f3d-420a-8af0-869cc7515c16 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:57.993471 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8b0dfd46-6f3d-420a-8af0-869cc7515c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:58.050919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1342/2670] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:58.060810 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:58.062226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:58.105405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:58.122275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515011f1-19f2-44c8-9e56-077a79900033 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:58.125959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22e4e9-8568-42d2-b97f-003dd8c3c4d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:58.126573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e4e9-8568-42d2-b97f-003dd8c3c4d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.126990 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e4e9-8568-42d2-b97f-003dd8c3c4d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.127262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e4e9-8568-42d2-b97f-003dd8c3c4d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.135942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22e4e9-8568-42d2-b97f-003dd8c3c4d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:58.137375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22e4e9-8568-42d2-b97f-003dd8c3c4d7 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:58.147337 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1329/2671] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:58.418655 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dfd46-6f3d-420a-8af0-869cc7515c16 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:58.420737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8b0dfd46-6f3d-420a-8af0-869cc7515c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:58.436073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dfd46-6f3d-420a-8af0-869cc7515c16 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:58.440812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:58.441416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.441691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.441996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] PATH_INFO: `/v3/projects/e19997fe7ee748d797eb3351a5ca119c` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.443189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19997fe7ee748d797eb3351a5ca119c)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:58.445105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:58.502144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19997fe7ee748d797eb3351a5ca119c', 'request_id': 'req-19073a1a-91a3-4669-953e-67d958e29658'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:58.508619 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19997fe7ee748d797eb3351a5ca119c', 'request_id': 'req-19073a1a-91a3-4669-953e-67d958e29658'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:58.518285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19997fe7ee748d797eb3351a5ca119c', 'request_id': 'req-19073a1a-91a3-4669-953e-67d958e29658'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:58.530500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] The token cache is being invalidate because project e19997fe7ee748d797eb3351a5ca119c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:58.530787 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:58.531448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-19073a1a-91a3-4669-953e-67d958e29658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:58.532927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1343/2672] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:58 2022] DELETE /identity/v3/projects/e19997fe7ee748d797eb3351a5ca119c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:58.537834 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d491a-40e4-4c6f-a550-44f2913dec8a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.538117 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d491a-40e4-4c6f-a550-44f2913dec8a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.538385 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d491a-40e4-4c6f-a550-44f2913dec8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.558318 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-045d491a-40e4-4c6f-a550-44f2913dec8a None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:58.560206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-045d491a-40e4-4c6f-a550-44f2913dec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:58.624714 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1330/2673] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:58.633584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181d0b1-98e2-4c69-8b7d-20c31fa9403d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.633867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181d0b1-98e2-4c69-8b7d-20c31fa9403d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.634133 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181d0b1-98e2-4c69-8b7d-20c31fa9403d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.655157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-7181d0b1-98e2-4c69-8b7d-20c31fa9403d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:58.656341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7181d0b1-98e2-4c69-8b7d-20c31fa9403d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:58.664453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1344/2674] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:58.668416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d491a-40e4-4c6f-a550-44f2913dec8a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:58.676796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181d0b1-98e2-4c69-8b7d-20c31fa9403d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:58.678291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d491a-40e4-4c6f-a550-44f2913dec8a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:58.678405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7181d0b1-98e2-4c69-8b7d-20c31fa9403d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:58.682849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf71cd-c163-4bd6-bf85-afb7261511e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:58.683439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf71cd-c163-4bd6-bf85-afb7261511e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.683710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf71cd-c163-4bd6-bf85-afb7261511e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.684254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf71cd-c163-4bd6-bf85-afb7261511e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.684936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf71cd-c163-4bd6-bf85-afb7261511e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:58.685264 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf71cd-c163-4bd6-bf85-afb7261511e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:58.686820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf71cd-c163-4bd6-bf85-afb7261511e1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:58.698638 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:58.698929 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1331/2675] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:58.703923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf71cd-c163-4bd6-bf85-afb7261511e1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:58.714227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf71cd-c163-4bd6-bf85-afb7261511e1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:58.719380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30d07-d985-4180-aa8c-96cca190c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:58.719953 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30d07-d985-4180-aa8c-96cca190c20c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.720227 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30d07-d985-4180-aa8c-96cca190c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.720489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30d07-d985-4180-aa8c-96cca190c20c admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.721134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e30d07-d985-4180-aa8c-96cca190c20c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:58.722996 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e30d07-d985-4180-aa8c-96cca190c20c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:58.740399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181d0b1-98e2-4c69-8b7d-20c31fa9403d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:58.744692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba38321-e84d-4b49-839a-f9cf5a30569d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:58.745399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba38321-e84d-4b49-839a-f9cf5a30569d service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.745678 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba38321-e84d-4b49-839a-f9cf5a30569d service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.745945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba38321-e84d-4b49-839a-f9cf5a30569d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.747766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-7ba38321-e84d-4b49-839a-f9cf5a30569d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:58.751070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1332/2676] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:58.755112 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30d07-d985-4180-aa8c-96cca190c20c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:58.812583 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba38321-e84d-4b49-839a-f9cf5a30569d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:58.814204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba38321-e84d-4b49-839a-f9cf5a30569d service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:58.818242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30d07-d985-4180-aa8c-96cca190c20c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:58.822942 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83514f-6498-4046-a08d-6eb1c9269a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:58.823246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1345/2677] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:08:58.823619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83514f-6498-4046-a08d-6eb1c9269a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.823904 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83514f-6498-4046-a08d-6eb1c9269a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.824175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83514f-6498-4046-a08d-6eb1c9269a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.824865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83514f-6498-4046-a08d-6eb1c9269a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:58.827621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83514f-6498-4046-a08d-6eb1c9269a0e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:58.851811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1333/2678] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:58 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:58.856146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba38321-e84d-4b49-839a-f9cf5a30569d service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:58.867114 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba38321-e84d-4b49-839a-f9cf5a30569d service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:58.871666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5772b-61db-4e35-adf3-8f088bf7ce54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:58.872319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5772b-61db-4e35-adf3-8f088bf7ce54 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.872587 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5772b-61db-4e35-adf3-8f088bf7ce54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.872850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5772b-61db-4e35-adf3-8f088bf7ce54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.873582 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5772b-61db-4e35-adf3-8f088bf7ce54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:58.875156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5772b-61db-4e35-adf3-8f088bf7ce54 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:58.884964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1346/2679] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:58.889943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83514f-6498-4046-a08d-6eb1c9269a0e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:58.900701 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83514f-6498-4046-a08d-6eb1c9269a0e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:58.906632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d52f0d-a2ee-4cfc-be25-6b71e9df008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:58.907220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d52f0d-a2ee-4cfc-be25-6b71e9df008c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.907504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d52f0d-a2ee-4cfc-be25-6b71e9df008c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.908181 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d52f0d-a2ee-4cfc-be25-6b71e9df008c admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.908772 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d52f0d-a2ee-4cfc-be25-6b71e9df008c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:58.910538 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d52f0d-a2ee-4cfc-be25-6b71e9df008c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:58.922639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1334/2680] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:58.927080 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5772b-61db-4e35-adf3-8f088bf7ce54 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:58.939120 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5772b-61db-4e35-adf3-8f088bf7ce54 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:58.944285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4239c-60e3-46ed-9277-fabdecc43e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:58.944956 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4239c-60e3-46ed-9277-fabdecc43e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.945262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4239c-60e3-46ed-9277-fabdecc43e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.945709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4239c-60e3-46ed-9277-fabdecc43e1e admin admin] PATH_INFO: `/v3/projects/aa4d639296c34f1dbf0b327fd0828620/users/21a84ede2a594f7ab23b442baa9f2f5a/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:58.961946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4239c-60e3-46ed-9277-fabdecc43e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4d639296c34f1dbf0b327fd0828620, user_id=21a84ede2a594f7ab23b442baa9f2f5a, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:58.964690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4239c-60e3-46ed-9277-fabdecc43e1e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:58.986136 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1347/2681] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:58 2022] PUT /identity/v3/projects/aa4d639296c34f1dbf0b327fd0828620/users/21a84ede2a594f7ab23b442baa9f2f5a/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:58.988004 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539010-e2a8-4289-8203-3d44b476977e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:58.988341 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539010-e2a8-4289-8203-3d44b476977e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:58.988619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539010-e2a8-4289-8203-3d44b476977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:59.008968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3c539010-e2a8-4289-8203-3d44b476977e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:59.010272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3c539010-e2a8-4289-8203-3d44b476977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:59.032889 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:59.084827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1335/2682] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:59.100276 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4239c-60e3-46ed-9277-fabdecc43e1e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:59.105056 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8ee4239c-60e3-46ed-9277-fabdecc43e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:59.181467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4239c-60e3-46ed-9277-fabdecc43e1e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:59.186202 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026309d-6016-42dc-88c3-1224089928ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:59.186909 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026309d-6016-42dc-88c3-1224089928ed service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:59.187284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026309d-6016-42dc-88c3-1224089928ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:59.187562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026309d-6016-42dc-88c3-1224089928ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:59.198439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026309d-6016-42dc-88c3-1224089928ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:59.199888 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026309d-6016-42dc-88c3-1224089928ed service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:59.211503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1348/2683] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:59.417037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c539010-e2a8-4289-8203-3d44b476977e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:59.477296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c539010-e2a8-4289-8203-3d44b476977e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:59.483543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:59.484323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:59.484589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:59.484859 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:59.486650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:59.572423 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:59.573939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:59.584204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1336/2684] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:00.756861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e052e4-41da-4ad1-81b4-641fdcc498a4 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:00.757439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e052e4-41da-4ad1-81b4-641fdcc498a4 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:00.757439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e052e4-41da-4ad1-81b4-641fdcc498a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:00.802475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-85e052e4-41da-4ad1-81b4-641fdcc498a4 None None] MFA Rules not processed for user `d467895d586a43719d2abdf4105187f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:00.803714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-85e052e4-41da-4ad1-81b4-641fdcc498a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:00.863596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:00.909388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1349/2685] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:01.158902 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:01.160681 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:01.172868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2956878-d704-4410-a5a9-e69f98267bc6 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:01.176543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911eb36-f09f-4c99-910d-0736bdcb9e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:01.177243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911eb36-f09f-4c99-910d-0736bdcb9e4f service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:01.177588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911eb36-f09f-4c99-910d-0736bdcb9e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:01.177975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911eb36-f09f-4c99-910d-0736bdcb9e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:01.179895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b911eb36-f09f-4c99-910d-0736bdcb9e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:01.191424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911eb36-f09f-4c99-910d-0736bdcb9e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:01.192822 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911eb36-f09f-4c99-910d-0736bdcb9e4f service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:01.204386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1337/2686] 10.222.0.78 () {60 vars in 1396 bytes} [Tue Aug 30 01:09:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:03.847149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e052e4-41da-4ad1-81b4-641fdcc498a4 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:03.848634 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-85e052e4-41da-4ad1-81b4-641fdcc498a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:03.879533 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e052e4-41da-4ad1-81b4-641fdcc498a4 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:03.900641 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:03.905358 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:03.905645 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:03.905913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] PATH_INFO: `/v3/users/55c3ede88baa4b8e95e7d4411c5cc6e3` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:03.909812 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c3ede88baa4b8e95e7d4411c5cc6e3)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:03.911763 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:04.022930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c3ede88baa4b8e95e7d4411c5cc6e3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:04.044301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c3ede88baa4b8e95e7d4411c5cc6e3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:04.066269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c3ede88baa4b8e95e7d4411c5cc6e3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:04.068426 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c3ede88baa4b8e95e7d4411c5cc6e3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:04.094361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c3ede88baa4b8e95e7d4411c5cc6e3'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:04.109069 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1350/2687] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:09:03 2022] DELETE /identity/v3/users/55c3ede88baa4b8e95e7d4411c5cc6e3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:04.115441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12353294-cdc4-47fd-8407-fd4a7353db52 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:04.117370 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12353294-cdc4-47fd-8407-fd4a7353db52 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:04.117826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12353294-cdc4-47fd-8407-fd4a7353db52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:04.189265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-12353294-cdc4-47fd-8407-fd4a7353db52 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:04.190724 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-12353294-cdc4-47fd-8407-fd4a7353db52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:04.368805 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1338/2688] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:04.388747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:04.390651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:04.523694 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ea4f-c6bd-42e0-9ada-35aad3fd30b7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:04.532155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95aebd-77c5-425f-a609-822f6f99df1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:04.533029 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95aebd-77c5-425f-a609-822f6f99df1f service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:04.533332 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95aebd-77c5-425f-a609-822f6f99df1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:04.533612 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95aebd-77c5-425f-a609-822f6f99df1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:04.549905 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95aebd-77c5-425f-a609-822f6f99df1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:04.553243 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95aebd-77c5-425f-a609-822f6f99df1f service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:04.581895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1351/2689] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:04.870658 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f34b9-c6fd-4a00-bc01-c8a17d7317bf None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:04.871027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f34b9-c6fd-4a00-bc01-c8a17d7317bf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:04.871372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f34b9-c6fd-4a00-bc01-c8a17d7317bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:04.887530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:04.979840 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1a6f34b9-c6fd-4a00-bc01-c8a17d7317bf None None] MFA Rules not processed for user `21a84ede2a594f7ab23b442baa9f2f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:04.987199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1a6f34b9-c6fd-4a00-bc01-c8a17d7317bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:05.156406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1339/2690] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:05.161046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95aebd-77c5-425f-a609-822f6f99df1f service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:05.162910 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8c95aebd-77c5-425f-a609-822f6f99df1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:05.209737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f34b9-c6fd-4a00-bc01-c8a17d7317bf None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:05.211373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1a6f34b9-c6fd-4a00-bc01-c8a17d7317bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:05.227473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95aebd-77c5-425f-a609-822f6f99df1f service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:05.235521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed35d1c-4b66-4238-832b-67765e4c5a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:05.236824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed35d1c-4b66-4238-832b-67765e4c5a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.237415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed35d1c-4b66-4238-832b-67765e4c5a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.237785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed35d1c-4b66-4238-832b-67765e4c5a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.238519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed35d1c-4b66-4238-832b-67765e4c5a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:05.238975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f34b9-c6fd-4a00-bc01-c8a17d7317bf None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:05.241393 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed35d1c-4b66-4238-832b-67765e4c5a0e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:05.251565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:05.252211 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.252493 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.252762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] PATH_INFO: `/v3/projects/7e68d4489efc47ea9aae9ee7d73ca5ca` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.254221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e68d4489efc47ea9aae9ee7d73ca5ca)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:05.256327 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:05.295010 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1352/2691] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:09:05 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:09:05.298403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed35d1c-4b66-4238-832b-67765e4c5a0e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:05.329603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e68d4489efc47ea9aae9ee7d73ca5ca', 'request_id': 'req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.338014 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e68d4489efc47ea9aae9ee7d73ca5ca', 'request_id': 'req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.352146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:05.355060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e68d4489efc47ea9aae9ee7d73ca5ca', 'request_id': 'req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.370288 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] The token cache is being invalidate because project 7e68d4489efc47ea9aae9ee7d73ca5ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:09:05.370857 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.371790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.377414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1340/2692] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:09:05 2022] DELETE /identity/v3/projects/7e68d4489efc47ea9aae9ee7d73ca5ca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:09:05.379634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:05.382660 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:05.386223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed35d1c-4b66-4238-832b-67765e4c5a0e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:05.480240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab992d-f2f6-4b49-9511-1b3c204b0581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:05.480846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab992d-f2f6-4b49-9511-1b3c204b0581 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.481828 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab992d-f2f6-4b49-9511-1b3c204b0581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.482110 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab992d-f2f6-4b49-9511-1b3c204b0581 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.482749 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab992d-f2f6-4b49-9511-1b3c204b0581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:05.484827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab992d-f2f6-4b49-9511-1b3c204b0581 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:05.510298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ee9b3-7057-4c6e-a1c2-3b4a45ad5449 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:05.537748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:05.538556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.540010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.540434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] PATH_INFO: `/v3/users/54c50f3163cb4c44a7b9ed04c2a84ca7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.547466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54c50f3163cb4c44a7b9ed04c2a84ca7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:05.556509 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:05.557628 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1353/2693] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:09:05 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:09:05.564361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab992d-f2f6-4b49-9511-1b3c204b0581 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:05.568207 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-61ab992d-f2f6-4b49-9511-1b3c204b0581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:05.596817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab992d-f2f6-4b49-9511-1b3c204b0581 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:05.599436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:05.618311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a2749-f9b8-48a9-9636-6177651d8b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:05.618945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a2749-f9b8-48a9-9636-6177651d8b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.619246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a2749-f9b8-48a9-9636-6177651d8b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.619502 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a2749-f9b8-48a9-9636-6177651d8b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.620159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a2749-f9b8-48a9-9636-6177651d8b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:05.627303 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a2749-f9b8-48a9-9636-6177651d8b3a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:05.646716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1354/2694] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:09:05.651584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a2749-f9b8-48a9-9636-6177651d8b3a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:05.667652 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a2749-f9b8-48a9-9636-6177651d8b3a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:05.674743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dabb24-9a21-4e23-a473-10cdc90f1ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:05.675679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dabb24-9a21-4e23-a473-10cdc90f1ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.676028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dabb24-9a21-4e23-a473-10cdc90f1ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.676340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dabb24-9a21-4e23-a473-10cdc90f1ee9 admin admin] PATH_INFO: `/v3/projects/0f2f7592e46b42e2bfcaadde11a53d56/users/f65ef4736d384d0ea7ef783a789296b0/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.740671 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dabb24-9a21-4e23-a473-10cdc90f1ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2f7592e46b42e2bfcaadde11a53d56, user_id=f65ef4736d384d0ea7ef783a789296b0, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:05.747727 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dabb24-9a21-4e23-a473-10cdc90f1ee9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:05.752563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54c50f3163cb4c44a7b9ed04c2a84ca7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.765559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54c50f3163cb4c44a7b9ed04c2a84ca7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.766245 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54c50f3163cb4c44a7b9ed04c2a84ca7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.767036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54c50f3163cb4c44a7b9ed04c2a84ca7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.775437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1355/2695] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:09:05 2022] PUT /identity/v3/projects/0f2f7592e46b42e2bfcaadde11a53d56/users/f65ef4736d384d0ea7ef783a789296b0/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:09:05.776874 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dabb24-9a21-4e23-a473-10cdc90f1ee9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:05.783560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-37dabb24-9a21-4e23-a473-10cdc90f1ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:05.785298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54c50f3163cb4c44a7b9ed04c2a84ca7'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:05.854416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1341/2696] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:09:05 2022] DELETE /identity/v3/users/54c50f3163cb4c44a7b9ed04c2a84ca7 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:05.952339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dabb24-9a21-4e23-a473-10cdc90f1ee9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:06.034281 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:06.062980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a66fa9-b43c-4cce-a46e-dcd2c1e87698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:06.063340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a66fa9-b43c-4cce-a46e-dcd2c1e87698 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:06.063571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a66fa9-b43c-4cce-a46e-dcd2c1e87698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:06.063840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a66fa9-b43c-4cce-a46e-dcd2c1e87698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:06.064738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a66fa9-b43c-4cce-a46e-dcd2c1e87698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:06.066401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a66fa9-b43c-4cce-a46e-dcd2c1e87698 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:06.080728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1356/2697] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:09:06.084857 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:06.109719 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee4be8-8c25-4447-be38-4cbf114a17a3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:06.118978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f3c35-e704-4c16-972d-a3eba43025af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:06.119854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f3c35-e704-4c16-972d-a3eba43025af admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:06.119986 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f3c35-e704-4c16-972d-a3eba43025af admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:06.120198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f3c35-e704-4c16-972d-a3eba43025af admin admin] PATH_INFO: `/v3/projects/0f2f7592e46b42e2bfcaadde11a53d56/users/f65ef4736d384d0ea7ef783a789296b0/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:06.123445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f3c35-e704-4c16-972d-a3eba43025af admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2f7592e46b42e2bfcaadde11a53d56, user_id=f65ef4736d384d0ea7ef783a789296b0, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:06.128008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f3c35-e704-4c16-972d-a3eba43025af admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:06.143242 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1342/2698] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:09:06 2022] PUT /identity/v3/projects/0f2f7592e46b42e2bfcaadde11a53d56/users/f65ef4736d384d0ea7ef783a789296b0/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:06.419182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a66fa9-b43c-4cce-a46e-dcd2c1e87698 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:06.421042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d0a66fa9-b43c-4cce-a46e-dcd2c1e87698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:06.442986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a66fa9-b43c-4cce-a46e-dcd2c1e87698 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:06.585109 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:06.585838 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:06.586185 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:06.586525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] PATH_INFO: `/v3/projects/0b551f16f8f44b6981be3b7a9cc3410d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:06.592537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b551f16f8f44b6981be3b7a9cc3410d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:06.593673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:06.706653 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b551f16f8f44b6981be3b7a9cc3410d', 'request_id': 'req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:06.734288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b551f16f8f44b6981be3b7a9cc3410d', 'request_id': 'req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:06.757375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b551f16f8f44b6981be3b7a9cc3410d', 'request_id': 'req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:06.776673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] The token cache is being invalidate because project 0b551f16f8f44b6981be3b7a9cc3410d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:09:06.776972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:06.780577 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:06.782204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1357/2699] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:09:06 2022] DELETE /identity/v3/projects/0b551f16f8f44b6981be3b7a9cc3410d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:06.786812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f3c35-e704-4c16-972d-a3eba43025af admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:06.789095 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bf4f3c35-e704-4c16-972d-a3eba43025af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:06.811838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f3c35-e704-4c16-972d-a3eba43025af admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:06.926082 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:06.927024 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:06.927479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:06.927911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] PATH_INFO: `/v3/users/4a155ef4562e460aa05d371d85aad746` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:06.930058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a155ef4562e460aa05d371d85aad746)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:06.933132 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:07.016983 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a155ef4562e460aa05d371d85aad746'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.032994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a155ef4562e460aa05d371d85aad746'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.034939 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a155ef4562e460aa05d371d85aad746'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.036130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a155ef4562e460aa05d371d85aad746'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.061148 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4ad11a6c-d397-4e2e-bc09-2df4882a0538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a155ef4562e460aa05d371d85aad746'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.073210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=19,keystone:SELECT=78,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:07.081304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1343/2700] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:09:06 2022] DELETE /identity/v3/users/4a155ef4562e460aa05d371d85aad746 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:07.405728 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:07.420917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ed3cb-cc85-4d66-aee0-ac6ebd5fb995 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:07.511479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f9183-f410-4cc7-8527-256bb4db99ac None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:07.511677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f9183-f410-4cc7-8527-256bb4db99ac None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:07.511865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f9183-f410-4cc7-8527-256bb4db99ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:07.574414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:07.595292 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6c7f9183-f410-4cc7-8527-256bb4db99ac None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:07.597065 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6c7f9183-f410-4cc7-8527-256bb4db99ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:07.622633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:07.623342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:07.623662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:07.623937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] PATH_INFO: `/v3/projects/ff65a6aad874428d8f3e1d4360104254` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:07.625259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff65a6aad874428d8f3e1d4360104254)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:07.627367 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:07.699206 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff65a6aad874428d8f3e1d4360104254', 'request_id': 'req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.709826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff65a6aad874428d8f3e1d4360104254', 'request_id': 'req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.724229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1344/2701] 10.222.0.78 () {58 vars in 967 bytes} [Tue Aug 30 01:09:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:09:07.728644 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff65a6aad874428d8f3e1d4360104254', 'request_id': 'req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.742141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f9183-f410-4cc7-8527-256bb4db99ac None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:07.744962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6c7f9183-f410-4cc7-8527-256bb4db99ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:07.760242 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] The token cache is being invalidate because project ff65a6aad874428d8f3e1d4360104254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:09:07.763408 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.764422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-057ddbb4-5f5d-42a3-85cb-b7f6a0e4a9ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:07.768226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1358/2702] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:09:07 2022] DELETE /identity/v3/projects/ff65a6aad874428d8f3e1d4360104254 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:07.773597 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e483f-5899-462f-a272-54bdfd4e7ff6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:07.773947 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e483f-5899-462f-a272-54bdfd4e7ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:07.774605 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e483f-5899-462f-a272-54bdfd4e7ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:07.887696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e17e483f-5899-462f-a272-54bdfd4e7ff6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:07.889970 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e17e483f-5899-462f-a272-54bdfd4e7ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:07.925766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f9183-f410-4cc7-8527-256bb4db99ac None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:07.996590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1359/2703] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:08.003988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992c372-65f3-4dc2-ad69-fe78f3de74ab None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.004496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992c372-65f3-4dc2-ad69-fe78f3de74ab None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.005041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992c372-65f3-4dc2-ad69-fe78f3de74ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.039238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8992c372-65f3-4dc2-ad69-fe78f3de74ab None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:08.040473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8992c372-65f3-4dc2-ad69-fe78f3de74ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:08.061526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c6206-4952-4175-95e5-7ea93e54cc96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:08.062319 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c6206-4952-4175-95e5-7ea93e54cc96 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.062897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1360/2704] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:09:08.063206 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c6206-4952-4175-95e5-7ea93e54cc96 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.063525 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c6206-4952-4175-95e5-7ea93e54cc96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.065480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c99c6206-4952-4175-95e5-7ea93e54cc96 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:08.067791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992c372-65f3-4dc2-ad69-fe78f3de74ab None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:08.081511 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992c372-65f3-4dc2-ad69-fe78f3de74ab None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:08.086783 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712548ce-0d89-4282-b9a1-4fbebe6c1d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:08.087453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712548ce-0d89-4282-b9a1-4fbebe6c1d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.087787 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712548ce-0d89-4282-b9a1-4fbebe6c1d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.088058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712548ce-0d89-4282-b9a1-4fbebe6c1d8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.088714 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712548ce-0d89-4282-b9a1-4fbebe6c1d8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:09:08.089065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712548ce-0d89-4282-b9a1-4fbebe6c1d8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:08.091413 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712548ce-0d89-4282-b9a1-4fbebe6c1d8e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:08.102700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:08.108197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1361/2705] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:09:08.113903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712548ce-0d89-4282-b9a1-4fbebe6c1d8e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:08.119936 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:08.132344 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712548ce-0d89-4282-b9a1-4fbebe6c1d8e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:08.147555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dda91c-8511-4189-a9df-6b0148b94abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:08.148176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dda91c-8511-4189-a9df-6b0148b94abd admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.148451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dda91c-8511-4189-a9df-6b0148b94abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.148719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dda91c-8511-4189-a9df-6b0148b94abd admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.149423 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dda91c-8511-4189-a9df-6b0148b94abd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:08.151277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dda91c-8511-4189-a9df-6b0148b94abd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:08.186428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c6206-4952-4175-95e5-7ea93e54cc96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:08.188156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c6206-4952-4175-95e5-7ea93e54cc96 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:08.212273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1345/2706] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:09:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:09:08.218690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1362/2707] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:09:08 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:09:08.220396 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c6206-4952-4175-95e5-7ea93e54cc96 service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:08.237800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c6206-4952-4175-95e5-7ea93e54cc96 service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:08.428609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670a3c-5788-4c3d-8f8a-0085c9de22ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:08.429578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670a3c-5788-4c3d-8f8a-0085c9de22ff admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.430016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670a3c-5788-4c3d-8f8a-0085c9de22ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.432448 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670a3c-5788-4c3d-8f8a-0085c9de22ff admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.433301 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9670a3c-5788-4c3d-8f8a-0085c9de22ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:08.439228 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9670a3c-5788-4c3d-8f8a-0085c9de22ff admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:08.492137 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1346/2708] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:09:08 2022] POST /identity/v3/users => generated 292 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:09:08.497099 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dda91c-8511-4189-a9df-6b0148b94abd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:08.521691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dda91c-8511-4189-a9df-6b0148b94abd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:08.541830 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78364d-3222-4a9a-a1dd-f0dc9721cdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:08.542463 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78364d-3222-4a9a-a1dd-f0dc9721cdc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.542748 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78364d-3222-4a9a-a1dd-f0dc9721cdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.547175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78364d-3222-4a9a-a1dd-f0dc9721cdc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.548137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78364d-3222-4a9a-a1dd-f0dc9721cdc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:08.551557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78364d-3222-4a9a-a1dd-f0dc9721cdc8 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:08.580228 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670a3c-5788-4c3d-8f8a-0085c9de22ff admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:08.581599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1363/2709] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:09:08.605205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670a3c-5788-4c3d-8f8a-0085c9de22ff admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:08.639984 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e5fc7-9a5f-4233-9327-55fd9d4e432e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:08.642044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e5fc7-9a5f-4233-9327-55fd9d4e432e admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.642548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e5fc7-9a5f-4233-9327-55fd9d4e432e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.647486 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e5fc7-9a5f-4233-9327-55fd9d4e432e admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.648980 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e5fc7-9a5f-4233-9327-55fd9d4e432e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:08.654533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e5fc7-9a5f-4233-9327-55fd9d4e432e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:08.717739 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78364d-3222-4a9a-a1dd-f0dc9721cdc8 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:08.719178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1347/2710] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:09:08.733599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78364d-3222-4a9a-a1dd-f0dc9721cdc8 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:08.749184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c424298-b6ba-4e14-9467-5d05aa549401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:08.750223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424298-b6ba-4e14-9467-5d05aa549401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.750666 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424298-b6ba-4e14-9467-5d05aa549401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.752435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424298-b6ba-4e14-9467-5d05aa549401 admin admin] PATH_INFO: `/v3/projects/3cad3c95e2fa45f7b3458bf2eebe541d/users/b1aeeea5e7144b098b4ad2ced09678b5/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.792448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c424298-b6ba-4e14-9467-5d05aa549401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cad3c95e2fa45f7b3458bf2eebe541d, user_id=b1aeeea5e7144b098b4ad2ced09678b5, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:08.799467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c424298-b6ba-4e14-9467-5d05aa549401 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:08.837454 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1364/2711] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:09:08 2022] PUT /identity/v3/projects/3cad3c95e2fa45f7b3458bf2eebe541d/users/b1aeeea5e7144b098b4ad2ced09678b5/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:08.850955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86683e1-7f04-44a7-b34a-b616971d3105 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.851307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86683e1-7f04-44a7-b34a-b616971d3105 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.851628 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86683e1-7f04-44a7-b34a-b616971d3105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.952238 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b86683e1-7f04-44a7-b34a-b616971d3105 None None] MFA Rules not processed for user `b1aeeea5e7144b098b4ad2ced09678b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:08.953571 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b86683e1-7f04-44a7-b34a-b616971d3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:09.154134 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1348/2712] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:09.160698 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c424298-b6ba-4e14-9467-5d05aa549401 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:09.286571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c424298-b6ba-4e14-9467-5d05aa549401 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:09.294804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a0371-055d-43aa-8f67-b3a96a7c4c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:09.295701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a0371-055d-43aa-8f67-b3a96a7c4c39 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:09.297028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a0371-055d-43aa-8f67-b3a96a7c4c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:09.297356 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a0371-055d-43aa-8f67-b3a96a7c4c39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:09.298014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a0371-055d-43aa-8f67-b3a96a7c4c39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:09.301223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a0371-055d-43aa-8f67-b3a96a7c4c39 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:09.352512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1365/2713] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:09:09 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:09:09.357415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86683e1-7f04-44a7-b34a-b616971d3105 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:09.372905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86683e1-7f04-44a7-b34a-b616971d3105 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:09.422192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:09.482207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2bfe2-ea0b-4936-a2cf-305ba6eaf639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:09.482914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2bfe2-ea0b-4936-a2cf-305ba6eaf639 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:09.483332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2bfe2-ea0b-4936-a2cf-305ba6eaf639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:09.483666 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2bfe2-ea0b-4936-a2cf-305ba6eaf639 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:09.484368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2bfe2-ea0b-4936-a2cf-305ba6eaf639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:09.486595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2bfe2-ea0b-4936-a2cf-305ba6eaf639 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:09.514565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1349/2714] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:09:09 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:09:09.515814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a0371-055d-43aa-8f67-b3a96a7c4c39 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:09.527111 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a0371-055d-43aa-8f67-b3a96a7c4c39 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:09.534160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4696767-8c4d-451c-a221-192256e2398e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:09.535009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4696767-8c4d-451c-a221-192256e2398e admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:09.535293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4696767-8c4d-451c-a221-192256e2398e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:09.535549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4696767-8c4d-451c-a221-192256e2398e admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:09.536340 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4696767-8c4d-451c-a221-192256e2398e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:09.538161 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4696767-8c4d-451c-a221-192256e2398e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:09.552962 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1366/2715] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:09:09.554648 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2bfe2-ea0b-4936-a2cf-305ba6eaf639 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:09.566173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2bfe2-ea0b-4936-a2cf-305ba6eaf639 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:09.571513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd155a-3ec8-4491-9e88-50967249d996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:09.571764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd155a-3ec8-4491-9e88-50967249d996 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:09.572140 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd155a-3ec8-4491-9e88-50967249d996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:09.572510 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd155a-3ec8-4491-9e88-50967249d996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:09.573307 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd155a-3ec8-4491-9e88-50967249d996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:09.575624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd155a-3ec8-4491-9e88-50967249d996 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:09.591485 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1350/2716] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:09:09.591710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4696767-8c4d-451c-a221-192256e2398e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:09.603297 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4696767-8c4d-451c-a221-192256e2398e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:09.608200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a1b51-ab1d-409c-a5ca-a4609b57fa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:09.609013 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a1b51-ab1d-409c-a5ca-a4609b57fa39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:09.609544 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a1b51-ab1d-409c-a5ca-a4609b57fa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:09.609914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a1b51-ab1d-409c-a5ca-a4609b57fa39 admin admin] PATH_INFO: `/v3/projects/22291b2a4b7b44adb836bd6e282fd2dd/users/22bdc2d6cf1b4ed8a7289a94d18c7a3d/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:09.631988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a1b51-ab1d-409c-a5ca-a4609b57fa39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22291b2a4b7b44adb836bd6e282fd2dd, user_id=22bdc2d6cf1b4ed8a7289a94d18c7a3d, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:09.636153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a1b51-ab1d-409c-a5ca-a4609b57fa39 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:09.664647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1367/2717] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:09:09 2022] PUT /identity/v3/projects/22291b2a4b7b44adb836bd6e282fd2dd/users/22bdc2d6cf1b4ed8a7289a94d18c7a3d/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:09:09.666144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397ec69-47d4-424d-b95c-f80210e9a3d9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:09.666718 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397ec69-47d4-424d-b95c-f80210e9a3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:09.666997 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397ec69-47d4-424d-b95c-f80210e9a3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:09.704234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3397ec69-47d4-424d-b95c-f80210e9a3d9 None None] MFA Rules not processed for user `22bdc2d6cf1b4ed8a7289a94d18c7a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:09.705959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3397ec69-47d4-424d-b95c-f80210e9a3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:09.827369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1351/2718] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:09.831907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a1b51-ab1d-409c-a5ca-a4609b57fa39 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:09.912440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:09.926722 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a1b51-ab1d-409c-a5ca-a4609b57fa39 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:09.933372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24792dd2-39ed-4fb9-a30b-f280a03980a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:09.936027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24792dd2-39ed-4fb9-a30b-f280a03980a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:09.936531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24792dd2-39ed-4fb9-a30b-f280a03980a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:09.937050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24792dd2-39ed-4fb9-a30b-f280a03980a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:09.937931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24792dd2-39ed-4fb9-a30b-f280a03980a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:09.940476 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24792dd2-39ed-4fb9-a30b-f280a03980a1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:09.979965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1368/2719] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:09:09 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:09:09.984751 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397ec69-47d4-424d-b95c-f80210e9a3d9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:10.004172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397ec69-47d4-424d-b95c-f80210e9a3d9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:10.080950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a30396-3ae7-49e6-98e7-2f331ec52475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:10.081744 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a30396-3ae7-49e6-98e7-2f331ec52475 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:10.082130 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a30396-3ae7-49e6-98e7-2f331ec52475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:10.082516 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a30396-3ae7-49e6-98e7-2f331ec52475 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:10.083285 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a30396-3ae7-49e6-98e7-2f331ec52475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:10.085329 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a30396-3ae7-49e6-98e7-2f331ec52475 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:10.135024 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1352/2720] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:09:09 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:09:10.142406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24792dd2-39ed-4fb9-a30b-f280a03980a1 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:10.155372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24792dd2-39ed-4fb9-a30b-f280a03980a1 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:10.161766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f378aa-2a75-4dd5-977d-2d99c6684555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:10.162424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f378aa-2a75-4dd5-977d-2d99c6684555 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:10.162637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f378aa-2a75-4dd5-977d-2d99c6684555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:10.162966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f378aa-2a75-4dd5-977d-2d99c6684555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:10.163617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f378aa-2a75-4dd5-977d-2d99c6684555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:10.165412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f378aa-2a75-4dd5-977d-2d99c6684555 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:10.181920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1369/2721] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:09:10.186513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a30396-3ae7-49e6-98e7-2f331ec52475 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:10.205229 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a30396-3ae7-49e6-98e7-2f331ec52475 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:10.211384 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aff13b-c1f2-41da-a35e-a33b110f026c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:10.212249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aff13b-c1f2-41da-a35e-a33b110f026c admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:10.212594 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aff13b-c1f2-41da-a35e-a33b110f026c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:10.212938 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aff13b-c1f2-41da-a35e-a33b110f026c admin admin] PATH_INFO: `/v3/projects/19273ac7bf7a45fa914c8f60fcdcbd4e/users/d460b0058221453ebf147b88a62c8ef9/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:10.242829 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aff13b-c1f2-41da-a35e-a33b110f026c admin admin] RBAC: Authorizing `identity:create_grant(project_id=19273ac7bf7a45fa914c8f60fcdcbd4e, user_id=d460b0058221453ebf147b88a62c8ef9, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:10.245976 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aff13b-c1f2-41da-a35e-a33b110f026c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:10.269055 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:10.280102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1353/2722] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:09:10 2022] PUT /identity/v3/projects/19273ac7bf7a45fa914c8f60fcdcbd4e/users/d460b0058221453ebf147b88a62c8ef9/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:10.284194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f378aa-2a75-4dd5-977d-2d99c6684555 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:10.298239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f378aa-2a75-4dd5-977d-2d99c6684555 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:10.469492 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986b07f-e4e3-4495-99de-90668e0e4ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:10.470270 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986b07f-e4e3-4495-99de-90668e0e4ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:10.470657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986b07f-e4e3-4495-99de-90668e0e4ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:10.471037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986b07f-e4e3-4495-99de-90668e0e4ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:10.471876 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986b07f-e4e3-4495-99de-90668e0e4ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:10.473697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986b07f-e4e3-4495-99de-90668e0e4ee9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:10.515764 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1370/2723] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:09:10.517378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aff13b-c1f2-41da-a35e-a33b110f026c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:10.536699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aff13b-c1f2-41da-a35e-a33b110f026c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:10.550562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363ec1f-7ced-4f1f-81fa-39dd59e2d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:10.551368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363ec1f-7ced-4f1f-81fa-39dd59e2d143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:10.553198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363ec1f-7ced-4f1f-81fa-39dd59e2d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:10.554603 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363ec1f-7ced-4f1f-81fa-39dd59e2d143 admin admin] PATH_INFO: `/v3/projects/19273ac7bf7a45fa914c8f60fcdcbd4e/users/d460b0058221453ebf147b88a62c8ef9/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:10.560864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363ec1f-7ced-4f1f-81fa-39dd59e2d143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19273ac7bf7a45fa914c8f60fcdcbd4e, user_id=d460b0058221453ebf147b88a62c8ef9, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:10.568072 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363ec1f-7ced-4f1f-81fa-39dd59e2d143 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:10.591184 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1354/2724] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:09:10 2022] PUT /identity/v3/projects/19273ac7bf7a45fa914c8f60fcdcbd4e/users/d460b0058221453ebf147b88a62c8ef9/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:10.602954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb22356-7111-4783-9a34-331428b145ae None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:10.603427 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb22356-7111-4783-9a34-331428b145ae None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:10.603657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb22356-7111-4783-9a34-331428b145ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:10.647289 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-dbb22356-7111-4783-9a34-331428b145ae None None] MFA Rules not processed for user `d460b0058221453ebf147b88a62c8ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:10.648620 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-dbb22356-7111-4783-9a34-331428b145ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:10.756424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1371/2725] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:10.771497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363ec1f-7ced-4f1f-81fa-39dd59e2d143 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:10.773591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2363ec1f-7ced-4f1f-81fa-39dd59e2d143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:10.881050 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363ec1f-7ced-4f1f-81fa-39dd59e2d143 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:10.886742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d2fb9-6414-41c5-bf61-8a7ee0d6407d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:10.888177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d2fb9-6414-41c5-bf61-8a7ee0d6407d service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:10.889042 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d2fb9-6414-41c5-bf61-8a7ee0d6407d service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:10.889380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d2fb9-6414-41c5-bf61-8a7ee0d6407d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:10.972869 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d2fb9-6414-41c5-bf61-8a7ee0d6407d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:10.974434 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d2fb9-6414-41c5-bf61-8a7ee0d6407d service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:10.987153 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1355/2726] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:12.709167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb22356-7111-4783-9a34-331428b145ae None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:12.710659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-dbb22356-7111-4783-9a34-331428b145ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:12.725152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb22356-7111-4783-9a34-331428b145ae None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:12.732063 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bef4cc-c527-4548-a70b-be9aaef33c02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:12.732490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bef4cc-c527-4548-a70b-be9aaef33c02 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:12.732765 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bef4cc-c527-4548-a70b-be9aaef33c02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:12.733065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bef4cc-c527-4548-a70b-be9aaef33c02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:12.843236 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:12.865759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bef4cc-c527-4548-a70b-be9aaef33c02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:12.869026 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bef4cc-c527-4548-a70b-be9aaef33c02 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:12.889863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1372/2727] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:13.033831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d106bcc-d845-4222-ab53-56d0b265d210 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:13.034371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d106bcc-d845-4222-ab53-56d0b265d210 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:13.034704 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d106bcc-d845-4222-ab53-56d0b265d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:13.070118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3d106bcc-d845-4222-ab53-56d0b265d210 None None] MFA Rules not processed for user `f65ef4736d384d0ea7ef783a789296b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:13.071706 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3d106bcc-d845-4222-ab53-56d0b265d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:13.184464 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1356/2728] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:13.196277 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bef4cc-c527-4548-a70b-be9aaef33c02 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:13.198156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-20bef4cc-c527-4548-a70b-be9aaef33c02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:13.328554 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bef4cc-c527-4548-a70b-be9aaef33c02 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:13.334717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7633936-a730-4b57-85dd-0b69d3f10387 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:13.335403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633936-a730-4b57-85dd-0b69d3f10387 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:13.335687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633936-a730-4b57-85dd-0b69d3f10387 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:13.335960 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633936-a730-4b57-85dd-0b69d3f10387 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:13.337786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e7633936-a730-4b57-85dd-0b69d3f10387 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:13.415760 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7633936-a730-4b57-85dd-0b69d3f10387 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:13.417799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7633936-a730-4b57-85dd-0b69d3f10387 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:13.429451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1373/2729] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:13.616428 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d106bcc-d845-4222-ab53-56d0b265d210 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:13.632262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d106bcc-d845-4222-ab53-56d0b265d210 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:13.637576 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc740d1-f269-41ba-b1be-9afa8ebd5e1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:13.638371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc740d1-f269-41ba-b1be-9afa8ebd5e1d service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:13.638560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc740d1-f269-41ba-b1be-9afa8ebd5e1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:13.639010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc740d1-f269-41ba-b1be-9afa8ebd5e1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:13.650151 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc740d1-f269-41ba-b1be-9afa8ebd5e1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:13.651669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc740d1-f269-41ba-b1be-9afa8ebd5e1d service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:13.672183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1357/2730] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:14.638350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcf268-4d2c-48ad-bcc7-d938ea385270 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:14.638917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcf268-4d2c-48ad-bcc7-d938ea385270 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:14.639138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcf268-4d2c-48ad-bcc7-d938ea385270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:14.660547 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-d0dcf268-4d2c-48ad-bcc7-d938ea385270 None None] MFA Rules not processed for user `21a84ede2a594f7ab23b442baa9f2f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:14.662283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-d0dcf268-4d2c-48ad-bcc7-d938ea385270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:14.670594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1374/2731] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:17.693333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc740d1-f269-41ba-b1be-9afa8ebd5e1d service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:17.708262 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:17.760885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc740d1-f269-41ba-b1be-9afa8ebd5e1d service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:17.765584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:17.766204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:17.766483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:17.766749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] PATH_INFO: `/v3/users/b1aeeea5e7144b098b4ad2ced09678b5` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:17.767865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1aeeea5e7144b098b4ad2ced09678b5)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:17.769825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:17.817965 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1aeeea5e7144b098b4ad2ced09678b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:17.826615 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1aeeea5e7144b098b4ad2ced09678b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:17.827232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1aeeea5e7144b098b4ad2ced09678b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:17.827874 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1aeeea5e7144b098b4ad2ced09678b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:17.840968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1aeeea5e7144b098b4ad2ced09678b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:17.853679 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1358/2732] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:09:17 2022] DELETE /identity/v3/users/b1aeeea5e7144b098b4ad2ced09678b5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:17.858965 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7db55a-786b-4207-9ffe-8e3ebb9bf627 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:17.859444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7db55a-786b-4207-9ffe-8e3ebb9bf627 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:17.859564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7db55a-786b-4207-9ffe-8e3ebb9bf627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:17.879165 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-af7db55a-786b-4207-9ffe-8e3ebb9bf627 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:17.880292 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-af7db55a-786b-4207-9ffe-8e3ebb9bf627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:17.934209 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:17.944907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1375/2733] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:17.955480 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:17.956985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:18.019837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5dfdb-63bf-4b09-b5ba-2cf44d7520f2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:18.023606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac9e71-ce70-4a7a-971b-9d47cd4124d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:18.024212 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac9e71-ce70-4a7a-971b-9d47cd4124d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:18.024500 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac9e71-ce70-4a7a-971b-9d47cd4124d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:18.024767 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac9e71-ce70-4a7a-971b-9d47cd4124d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:18.035372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac9e71-ce70-4a7a-971b-9d47cd4124d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:18.036821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac9e71-ce70-4a7a-971b-9d47cd4124d8 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:18.047758 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1359/2734] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:18.377707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7db55a-786b-4207-9ffe-8e3ebb9bf627 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:18.379197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-af7db55a-786b-4207-9ffe-8e3ebb9bf627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:18.396603 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7db55a-786b-4207-9ffe-8e3ebb9bf627 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:18.405843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:18.406551 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:18.406833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:18.407100 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] PATH_INFO: `/v3/projects/3cad3c95e2fa45f7b3458bf2eebe541d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:18.408636 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cad3c95e2fa45f7b3458bf2eebe541d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:18.410622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:18.475046 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cad3c95e2fa45f7b3458bf2eebe541d', 'request_id': 'req-416d507b-2d4c-4de9-bab2-786ab2754799'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.482331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cad3c95e2fa45f7b3458bf2eebe541d', 'request_id': 'req-416d507b-2d4c-4de9-bab2-786ab2754799'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.491316 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cad3c95e2fa45f7b3458bf2eebe541d', 'request_id': 'req-416d507b-2d4c-4de9-bab2-786ab2754799'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.506937 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] The token cache is being invalidate because project 3cad3c95e2fa45f7b3458bf2eebe541d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:09:18.507300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.507967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.509647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1376/2735] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:09:18 2022] DELETE /identity/v3/projects/3cad3c95e2fa45f7b3458bf2eebe541d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:18.515109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac9e71-ce70-4a7a-971b-9d47cd4124d8 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:18.525029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac9e71-ce70-4a7a-971b-9d47cd4124d8 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:18.559909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:18.587995 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:18.588621 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:18.588898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:18.589204 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] PATH_INFO: `/v3/users/22bdc2d6cf1b4ed8a7289a94d18c7a3d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:18.590610 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22bdc2d6cf1b4ed8a7289a94d18c7a3d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:18.592532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:18.646252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22bdc2d6cf1b4ed8a7289a94d18c7a3d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.655435 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22bdc2d6cf1b4ed8a7289a94d18c7a3d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.656090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22bdc2d6cf1b4ed8a7289a94d18c7a3d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.656723 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22bdc2d6cf1b4ed8a7289a94d18c7a3d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.669260 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22bdc2d6cf1b4ed8a7289a94d18c7a3d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:18.682698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1360/2736] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:09:18 2022] DELETE /identity/v3/users/22bdc2d6cf1b4ed8a7289a94d18c7a3d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:18.984170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:18.985712 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:19.002432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d507b-2d4c-4de9-bab2-786ab2754799 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:19.068983 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:19.069642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:19.069923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:19.070190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] PATH_INFO: `/v3/projects/22291b2a4b7b44adb836bd6e282fd2dd` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:19.071304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=22291b2a4b7b44adb836bd6e282fd2dd)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:19.073247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:19.123488 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22291b2a4b7b44adb836bd6e282fd2dd', 'request_id': 'req-b174232e-7470-41ec-9514-db82f1c47f4e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.130781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22291b2a4b7b44adb836bd6e282fd2dd', 'request_id': 'req-b174232e-7470-41ec-9514-db82f1c47f4e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.140113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22291b2a4b7b44adb836bd6e282fd2dd', 'request_id': 'req-b174232e-7470-41ec-9514-db82f1c47f4e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.153458 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] The token cache is being invalidate because project 22291b2a4b7b44adb836bd6e282fd2dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:09:19.153754 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.154442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.155972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1377/2737] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:09:18 2022] DELETE /identity/v3/projects/22291b2a4b7b44adb836bd6e282fd2dd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:19.160012 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:19.161483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:19.174635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274e3b1-d4d0-415b-b978-6ad13921df63 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:19.248189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:19.248943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:19.249271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:19.249543 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] PATH_INFO: `/v3/users/d460b0058221453ebf147b88a62c8ef9` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:19.250693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d460b0058221453ebf147b88a62c8ef9)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:19.252726 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:19.307099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd460b0058221453ebf147b88a62c8ef9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.322513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd460b0058221453ebf147b88a62c8ef9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.323275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd460b0058221453ebf147b88a62c8ef9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.325186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd460b0058221453ebf147b88a62c8ef9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.327354 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:19.344136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd460b0058221453ebf147b88a62c8ef9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.356747 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1361/2738] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:09:19 2022] DELETE /identity/v3/users/d460b0058221453ebf147b88a62c8ef9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:09:19.542214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:19.544149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:19.567095 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:19.658338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174232e-7470-41ec-9514-db82f1c47f4e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:19.689106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:19.690632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:19.729790 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e42a7b-d241-4c33-9540-46e3f147e767 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:19.731155 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e42a7b-d241-4c33-9540-46e3f147e767 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:19.731506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e42a7b-d241-4c33-9540-46e3f147e767 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:19.731835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e42a7b-d241-4c33-9540-46e3f147e767 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:19.734037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f3e42a7b-d241-4c33-9540-46e3f147e767 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:19.746843 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e42a7b-d241-4c33-9540-46e3f147e767 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:19.748464 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e42a7b-d241-4c33-9540-46e3f147e767 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:19.759092 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1378/2739] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:19.773444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116df03f-38c4-4f18-8208-8c4fecad8530 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:19.778659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:19.779291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:19.779565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:19.779858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] PATH_INFO: `/v3/projects/19273ac7bf7a45fa914c8f60fcdcbd4e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:19.782031 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19273ac7bf7a45fa914c8f60fcdcbd4e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:19.783933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:19.837217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19273ac7bf7a45fa914c8f60fcdcbd4e', 'request_id': 'req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.843581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19273ac7bf7a45fa914c8f60fcdcbd4e', 'request_id': 'req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.852366 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19273ac7bf7a45fa914c8f60fcdcbd4e', 'request_id': 'req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.865455 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] The token cache is being invalidate because project 19273ac7bf7a45fa914c8f60fcdcbd4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:09:19.865776 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.866578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cf7ffd67-ea9b-4bf6-a4ae-e626adbde599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:19.868377 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1362/2740] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:09:19 2022] DELETE /identity/v3/projects/19273ac7bf7a45fa914c8f60fcdcbd4e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:09:19.874326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160740b-4179-45f7-81b0-baa128154457 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:19.874741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160740b-4179-45f7-81b0-baa128154457 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:19.875096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160740b-4179-45f7-81b0-baa128154457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:19.894730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e160740b-4179-45f7-81b0-baa128154457 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:19.896088 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e160740b-4179-45f7-81b0-baa128154457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:19.963325 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1379/2741] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:19.973386 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f06f25-4121-4c57-b190-5ae384febb1e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:19.973676 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f06f25-4121-4c57-b190-5ae384febb1e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:19.973956 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f06f25-4121-4c57-b190-5ae384febb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:19.994466 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-51f06f25-4121-4c57-b190-5ae384febb1e None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:19.995661 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-51f06f25-4121-4c57-b190-5ae384febb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:20.006175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1363/2742] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:09:20.010591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160740b-4179-45f7-81b0-baa128154457 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:20.020305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160740b-4179-45f7-81b0-baa128154457 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:20.024819 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57730d99-d552-407d-8e46-215f72942a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:20.025445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57730d99-d552-407d-8e46-215f72942a18 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:20.025723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57730d99-d552-407d-8e46-215f72942a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:20.025988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57730d99-d552-407d-8e46-215f72942a18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:20.026618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57730d99-d552-407d-8e46-215f72942a18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:09:20.026914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57730d99-d552-407d-8e46-215f72942a18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:20.029255 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57730d99-d552-407d-8e46-215f72942a18 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:20.040440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1380/2743] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:09:20.045606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f06f25-4121-4c57-b190-5ae384febb1e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:20.056016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f06f25-4121-4c57-b190-5ae384febb1e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:20.060991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9ded6-4523-4a6a-8ae8-db8d149ff623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:20.061646 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9ded6-4523-4a6a-8ae8-db8d149ff623 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:20.061923 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9ded6-4523-4a6a-8ae8-db8d149ff623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:20.062191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9ded6-4523-4a6a-8ae8-db8d149ff623 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:20.062826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9ded6-4523-4a6a-8ae8-db8d149ff623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:20.064743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9ded6-4523-4a6a-8ae8-db8d149ff623 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:20.092164 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1364/2744] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:09:20 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:09:20.096902 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57730d99-d552-407d-8e46-215f72942a18 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:20.108623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57730d99-d552-407d-8e46-215f72942a18 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:20.116339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:20.167723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8c5de-8434-4e9e-92a8-b8f24ec4fc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:20.168839 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8c5de-8434-4e9e-92a8-b8f24ec4fc5a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:20.169146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8c5de-8434-4e9e-92a8-b8f24ec4fc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:20.169412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8c5de-8434-4e9e-92a8-b8f24ec4fc5a admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:20.170035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8c5de-8434-4e9e-92a8-b8f24ec4fc5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:20.171926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8c5de-8434-4e9e-92a8-b8f24ec4fc5a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:20.192968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1381/2745] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:09:20 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:09:20.197143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9ded6-4523-4a6a-8ae8-db8d149ff623 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:20.207788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9ded6-4523-4a6a-8ae8-db8d149ff623 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:20.212451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00766280-1217-4e84-b38d-8ad876b9eef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:20.213045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00766280-1217-4e84-b38d-8ad876b9eef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:20.213358 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00766280-1217-4e84-b38d-8ad876b9eef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:20.213630 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00766280-1217-4e84-b38d-8ad876b9eef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:20.214273 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00766280-1217-4e84-b38d-8ad876b9eef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:20.216469 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00766280-1217-4e84-b38d-8ad876b9eef0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:20.226958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1365/2746] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:09:20.231996 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8c5de-8434-4e9e-92a8-b8f24ec4fc5a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:20.243031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8c5de-8434-4e9e-92a8-b8f24ec4fc5a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:20.248211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c34c6-1922-493d-9056-2b3223d0708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:20.248840 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c34c6-1922-493d-9056-2b3223d0708f admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:20.249159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c34c6-1922-493d-9056-2b3223d0708f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:20.249497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c34c6-1922-493d-9056-2b3223d0708f admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:20.250150 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c34c6-1922-493d-9056-2b3223d0708f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:20.251747 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c34c6-1922-493d-9056-2b3223d0708f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:20.266210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1382/2747] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:09:20.270353 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00766280-1217-4e84-b38d-8ad876b9eef0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:20.281255 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00766280-1217-4e84-b38d-8ad876b9eef0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:20.286275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b9cfd-8886-4695-9fb5-8b9a5326955e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:20.286919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b9cfd-8886-4695-9fb5-8b9a5326955e admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:20.287201 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b9cfd-8886-4695-9fb5-8b9a5326955e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:20.287481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b9cfd-8886-4695-9fb5-8b9a5326955e admin admin] PATH_INFO: `/v3/projects/102e81ab843c49f6b563ba3c6c04a072/users/1cff6fe7bc204f8fb8530c300e82cae3/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:20.309862 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b9cfd-8886-4695-9fb5-8b9a5326955e admin admin] RBAC: Authorizing `identity:create_grant(project_id=102e81ab843c49f6b563ba3c6c04a072, user_id=1cff6fe7bc204f8fb8530c300e82cae3, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:20.312573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b9cfd-8886-4695-9fb5-8b9a5326955e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:20.323359 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:20.339974 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1366/2748] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:09:20 2022] PUT /identity/v3/projects/102e81ab843c49f6b563ba3c6c04a072/users/1cff6fe7bc204f8fb8530c300e82cae3/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:09:20.341750 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0550591-dcc6-4c52-993d-f8412f6fa1e3 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:20.341994 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0550591-dcc6-4c52-993d-f8412f6fa1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:20.342315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0550591-dcc6-4c52-993d-f8412f6fa1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:20.374182 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c0550591-dcc6-4c52-993d-f8412f6fa1e3 None None] MFA Rules not processed for user `1cff6fe7bc204f8fb8530c300e82cae3`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:20.375416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c0550591-dcc6-4c52-993d-f8412f6fa1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:20.449738 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1383/2749] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:09:20.460858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b9cfd-8886-4695-9fb5-8b9a5326955e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:20.462600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f21b9cfd-8886-4695-9fb5-8b9a5326955e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:20.531855 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b9cfd-8886-4695-9fb5-8b9a5326955e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:20.535581 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8c604-37fe-4c79-876c-759cb33d2988 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a47e77cc1942618a3584862042c5f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:20.536185 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8c604-37fe-4c79-876c-759cb33d2988 service glance] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:20.536461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8c604-37fe-4c79-876c-759cb33d2988 service glance] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:20.536730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8c604-37fe-4c79-876c-759cb33d2988 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:20.545672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8c604-37fe-4c79-876c-759cb33d2988 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:20.547086 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8c604-37fe-4c79-876c-759cb33d2988 service glance] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:20.555720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1367/2750] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:30.445842 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:30.551748 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=26 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:56.150049 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0550591-dcc6-4c52-993d-f8412f6fa1e3 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:56.151724 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c0550591-dcc6-4c52-993d-f8412f6fa1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:56.229627 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0550591-dcc6-4c52-993d-f8412f6fa1e3 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:56.234066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3222e-a8f8-47c6-8597-759d09d0c8c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:56.234662 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3222e-a8f8-47c6-8597-759d09d0c8c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:56.234938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3222e-a8f8-47c6-8597-759d09d0c8c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:56.235203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3222e-a8f8-47c6-8597-759d09d0c8c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:56.236921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8bb3222e-a8f8-47c6-8597-759d09d0c8c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:56.301301 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3222e-a8f8-47c6-8597-759d09d0c8c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:56.302741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3222e-a8f8-47c6-8597-759d09d0c8c9 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:56.312381 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1384/2751] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:58.274786 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8c604-37fe-4c79-876c-759cb33d2988 service glance] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:58.348370 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8c604-37fe-4c79-876c-759cb33d2988 service glance] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:58.353120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:58.353585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:58.353864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:58.354135 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] PATH_INFO: `/v3/users/1cff6fe7bc204f8fb8530c300e82cae3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:58.355275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cff6fe7bc204f8fb8530c300e82cae3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:58.357611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:58.412634 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cff6fe7bc204f8fb8530c300e82cae3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:58.420843 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cff6fe7bc204f8fb8530c300e82cae3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:58.421481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cff6fe7bc204f8fb8530c300e82cae3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:58.422143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cff6fe7bc204f8fb8530c300e82cae3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:58.438364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cff6fe7bc204f8fb8530c300e82cae3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:58.453912 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1368/2752] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:09:58 2022] DELETE /identity/v3/users/1cff6fe7bc204f8fb8530c300e82cae3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:09:58.459156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1404-e588-4aad-9504-aec9e76d173f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:58.459697 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1404-e588-4aad-9504-aec9e76d173f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:58.459810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1404-e588-4aad-9504-aec9e76d173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:58.478949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-c3ca1404-e588-4aad-9504-aec9e76d173f None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:58.480771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c3ca1404-e588-4aad-9504-aec9e76d173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:58.551711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1385/2753] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:58.564541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:58.565991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:58.633203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ed82a-07f8-43b5-a930-9681bfb33206 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:58.638667 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831d44f-01c8-4868-b341-b532ecc1d3e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:58.639354 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831d44f-01c8-4868-b341-b532ecc1d3e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:58.639698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831d44f-01c8-4868-b341-b532ecc1d3e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:58.640030 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831d44f-01c8-4868-b341-b532ecc1d3e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:58.652180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831d44f-01c8-4868-b341-b532ecc1d3e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:58.653913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831d44f-01c8-4868-b341-b532ecc1d3e4 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:58.664103 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1369/2754] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:59.000341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1404-e588-4aad-9504-aec9e76d173f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:59.002096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c3ca1404-e588-4aad-9504-aec9e76d173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:59.016832 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1404-e588-4aad-9504-aec9e76d173f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:59.030654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:59.031371 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:59.031562 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:59.032253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] PATH_INFO: `/v3/projects/102e81ab843c49f6b563ba3c6c04a072` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:59.033450 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=102e81ab843c49f6b563ba3c6c04a072)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:59.035412 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:59.067792 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:59.096188 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '102e81ab843c49f6b563ba3c6c04a072', 'request_id': 'req-76403462-7b5e-4632-ade5-d74c4f74ed88'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:59.103469 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '102e81ab843c49f6b563ba3c6c04a072', 'request_id': 'req-76403462-7b5e-4632-ade5-d74c4f74ed88'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:59.113485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '102e81ab843c49f6b563ba3c6c04a072', 'request_id': 'req-76403462-7b5e-4632-ade5-d74c4f74ed88'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:59.127512 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] The token cache is being invalidate because project 102e81ab843c49f6b563ba3c6c04a072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:09:59.127805 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:59.128615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:09:59.137239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1386/2755] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:09:58 2022] DELETE /identity/v3/projects/102e81ab843c49f6b563ba3c6c04a072 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:06.823541 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74194c-881a-4e34-b226-a40c62e8cce4 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:06.824157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74194c-881a-4e34-b226-a40c62e8cce4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:06.824157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74194c-881a-4e34-b226-a40c62e8cce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:06.844952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0f74194c-881a-4e34-b226-a40c62e8cce4 None None] MFA Rules not processed for user `3ff0c4a8fd8d44bda56ceeb47cd49abf`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:06.846205 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0f74194c-881a-4e34-b226-a40c62e8cce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:06.863606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:06.907085 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1370/2756] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:10:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:09.124549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:16.897517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=15 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:10.548555 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:10.549918 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:10.623660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76403462-7b5e-4632-ade5-d74c4f74ed88 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:10.627404 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:10.628160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:10.628447 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:10.628719 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:10.630669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:10.713157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:10.715631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:10.727606 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1387/2757] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:11.421770 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6133ffa-e022-41be-9bbb-d811b359666b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:11.422305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6133ffa-e022-41be-9bbb-d811b359666b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:11.422437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6133ffa-e022-41be-9bbb-d811b359666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:11.452962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d6133ffa-e022-41be-9bbb-d811b359666b None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:11.453968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d6133ffa-e022-41be-9bbb-d811b359666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:11.511281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1371/2758] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:11:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:11.563757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:11.565275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:11.639145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71567d41-5b83-49bd-b2cf-f15ef16ec81d service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:11.644232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44df45-8bda-4213-9f22-93f7f49b5578 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:11.644833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44df45-8bda-4213-9f22-93f7f49b5578 service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:11.645130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44df45-8bda-4213-9f22-93f7f49b5578 service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:11.645398 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44df45-8bda-4213-9f22-93f7f49b5578 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:11.654632 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44df45-8bda-4213-9f22-93f7f49b5578 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:11.656269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44df45-8bda-4213-9f22-93f7f49b5578 service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:11.664631 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1388/2759] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:11:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:11:15.557027 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6133ffa-e022-41be-9bbb-d811b359666b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:15.568317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6133ffa-e022-41be-9bbb-d811b359666b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:15.572159 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b9c31-4ae9-4a66-a1db-b1bb404c8ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:15.572771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b9c31-4ae9-4a66-a1db-b1bb404c8ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:15.573139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b9c31-4ae9-4a66-a1db-b1bb404c8ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:15.573413 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b9c31-4ae9-4a66-a1db-b1bb404c8ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:15.576782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6b2b9c31-4ae9-4a66-a1db-b1bb404c8ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:15.590970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b9c31-4ae9-4a66-a1db-b1bb404c8ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:15.592403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b9c31-4ae9-4a66-a1db-b1bb404c8ee0 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:15.606909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1372/2760] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:21.661290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=86 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:25.601346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=35 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:28.262649 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fe430-7691-4459-9b68-b62400119769 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:28.263173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fe430-7691-4459-9b68-b62400119769 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:28.263173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fe430-7691-4459-9b68-b62400119769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:28.319578 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f61fe430-7691-4459-9b68-b62400119769 None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:28.320817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f61fe430-7691-4459-9b68-b62400119769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:28.338716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1389/2761] 10.222.0.78 () {58 vars in 1009 bytes} [Tue Aug 30 01:11:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:28.351652 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b9c31-4ae9-4a66-a1db-b1bb404c8ee0 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:28.376265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b9c31-4ae9-4a66-a1db-b1bb404c8ee0 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:28.381348 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6290b6d-1e8e-47a8-8bfd-16a2e2721a7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:28.381947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6290b6d-1e8e-47a8-8bfd-16a2e2721a7a service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:28.382221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6290b6d-1e8e-47a8-8bfd-16a2e2721a7a service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:28.382488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6290b6d-1e8e-47a8-8bfd-16a2e2721a7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:28.392161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6290b6d-1e8e-47a8-8bfd-16a2e2721a7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:28.393757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6290b6d-1e8e-47a8-8bfd-16a2e2721a7a service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:28.403124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1373/2762] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:11:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:11:31.042505 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fe430-7691-4459-9b68-b62400119769 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:31.044145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f61fe430-7691-4459-9b68-b62400119769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:31.058096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fe430-7691-4459-9b68-b62400119769 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:31.063064 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:31.063827 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:31.064175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:31.064514 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:31.066732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:31.143989 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:31.145420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:31.156808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1390/2763] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:35.233978 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6290b6d-1e8e-47a8-8bfd-16a2e2721a7a service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:35.235585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f6290b6d-1e8e-47a8-8bfd-16a2e2721a7a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:35.390508 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6290b6d-1e8e-47a8-8bfd-16a2e2721a7a service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:35.395686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95bad0-e6ee-4022-83d8-a06fe4d7c62d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.396686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95bad0-e6ee-4022-83d8-a06fe4d7c62d service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.396968 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95bad0-e6ee-4022-83d8-a06fe4d7c62d service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.397265 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95bad0-e6ee-4022-83d8-a06fe4d7c62d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.399056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3f95bad0-e6ee-4022-83d8-a06fe4d7c62d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:35.417271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95bad0-e6ee-4022-83d8-a06fe4d7c62d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.419316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95bad0-e6ee-4022-83d8-a06fe4d7c62d service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.435158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1374/2764] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:38.703375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:38.704848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:38.719240 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda48898-7fe4-4145-ad0b-74d25fc02434 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:38.723884 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:38.724506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:38.724786 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:38.725057 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] PATH_INFO: `/v3/users/d467895d586a43719d2abdf4105187f8` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:38.726200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d467895d586a43719d2abdf4105187f8)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:38.728586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:38.776040 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd467895d586a43719d2abdf4105187f8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.783311 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd467895d586a43719d2abdf4105187f8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.790201 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd467895d586a43719d2abdf4105187f8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.790893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd467895d586a43719d2abdf4105187f8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.802439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd467895d586a43719d2abdf4105187f8'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.808442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1391/2765] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:38 2022] DELETE /identity/v3/users/d467895d586a43719d2abdf4105187f8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:38.990739 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95bad0-e6ee-4022-83d8-a06fe4d7c62d service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:39.002215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95bad0-e6ee-4022-83d8-a06fe4d7c62d service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:39.060317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:39.060832 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:39.061136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:39.061406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] PATH_INFO: `/v3/projects/a6380baa172744fa9d7ba5b81f847906` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:39.062523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6380baa172744fa9d7ba5b81f847906)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:39.064401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:39.110868 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6380baa172744fa9d7ba5b81f847906', 'request_id': 'req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.116885 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6380baa172744fa9d7ba5b81f847906', 'request_id': 'req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.125686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6380baa172744fa9d7ba5b81f847906', 'request_id': 'req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.132070 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:39.137946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] The token cache is being invalidate because project a6380baa172744fa9d7ba5b81f847906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:39.138244 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.139157 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.141333 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1375/2766] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:38 2022] DELETE /identity/v3/projects/a6380baa172744fa9d7ba5b81f847906 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:39.145491 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:39.147283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:39.160667 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349461-a3e6-49ba-8849-0b4c6eab04f3 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:39.184784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:39.217981 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:39.218596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:39.218873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:39.219139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] PATH_INFO: `/v3/users/a2162bb4321b44e48f56d41ddec1abfb` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:39.220261 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2162bb4321b44e48f56d41ddec1abfb)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:39.222196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:39.271462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2162bb4321b44e48f56d41ddec1abfb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.278695 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2162bb4321b44e48f56d41ddec1abfb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.285691 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2162bb4321b44e48f56d41ddec1abfb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.286312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2162bb4321b44e48f56d41ddec1abfb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.299115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2162bb4321b44e48f56d41ddec1abfb'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.304969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1392/2767] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:39 2022] DELETE /identity/v3/users/a2162bb4321b44e48f56d41ddec1abfb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:39.483186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:39.484783 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:39.501317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d0cb8-c178-44f6-9fef-df4b00f641b6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:39.573982 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:39.574731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:39.574951 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:39.575606 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] PATH_INFO: `/v3/projects/108bfe2bf3ef4249bb1f940b4ddbd8c6` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:39.576415 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108bfe2bf3ef4249bb1f940b4ddbd8c6)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:39.578685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:39.637622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108bfe2bf3ef4249bb1f940b4ddbd8c6', 'request_id': 'req-16478708-1d3b-4ec8-8598-6188dec96743'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.646215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108bfe2bf3ef4249bb1f940b4ddbd8c6', 'request_id': 'req-16478708-1d3b-4ec8-8598-6188dec96743'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.655062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108bfe2bf3ef4249bb1f940b4ddbd8c6', 'request_id': 'req-16478708-1d3b-4ec8-8598-6188dec96743'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.668220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] The token cache is being invalidate because project 108bfe2bf3ef4249bb1f940b4ddbd8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:39.668499 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.669179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16478708-1d3b-4ec8-8598-6188dec96743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.685110 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1376/2768] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:39 2022] DELETE /identity/v3/projects/108bfe2bf3ef4249bb1f940b4ddbd8c6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:41.769347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:41.771061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:41.827811 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3d346-a9c3-435c-8a95-7ce7db334b88 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:41.875841 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:41.876446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:41.876725 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:41.876992 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:41.878758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:41.889050 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:41.890510 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:41.898875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1393/2769] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:43.437147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29b2b5-d1ba-4481-ab50-319fa8d75bcf None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:43.437659 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29b2b5-d1ba-4481-ab50-319fa8d75bcf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:43.437769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29b2b5-d1ba-4481-ab50-319fa8d75bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:43.457109 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-af29b2b5-d1ba-4481-ab50-319fa8d75bcf None None] MFA Rules not processed for user `308bbd2bc1224533b573c80f0565252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:43.458303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-af29b2b5-d1ba-4481-ab50-319fa8d75bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:43.512593 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1377/2770] 10.222.0.78 () {58 vars in 967 bytes} [Tue Aug 30 01:11:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:43.525882 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:43.527278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:43.531501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:43.610035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895eef6c-7e25-48bf-b7de-00a5baf898a7 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:43.614898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620be8a-8d78-4151-aed2-f71af5f3dd1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53b9bcd675940c19febbc8fefb1f28b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:43.615519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620be8a-8d78-4151-aed2-f71af5f3dd1e service swift] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:43.615789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620be8a-8d78-4151-aed2-f71af5f3dd1e service swift] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:43.616051 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620be8a-8d78-4151-aed2-f71af5f3dd1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:43.623817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620be8a-8d78-4151-aed2-f71af5f3dd1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:43.625422 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620be8a-8d78-4151-aed2-f71af5f3dd1e service swift] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:43.633204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1394/2771] 10.222.0.78 () {60 vars in 1384 bytes} [Tue Aug 30 01:11:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:11:53.505780 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:53.630055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=35 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:11.158367 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd30c4-0b73-42c0-824a-7fd7dfdf077a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:11.158958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd30c4-0b73-42c0-824a-7fd7dfdf077a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:11.158958 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd30c4-0b73-42c0-824a-7fd7dfdf077a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:11.160555 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1378/2772] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:12:11.166683 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717f8ea-489f-43c6-bf68-9d7dc169cc9f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:11.167196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717f8ea-489f-43c6-bf68-9d7dc169cc9f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:11.167637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717f8ea-489f-43c6-bf68-9d7dc169cc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:11.206214 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9717f8ea-489f-43c6-bf68-9d7dc169cc9f None None] MFA Rules not processed for user `1a5d841380c243209f55314b545ff56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:11.207456 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9717f8ea-489f-43c6-bf68-9d7dc169cc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:11.214479 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1395/2773] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:12:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:12:11.226502 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd30c4-0b73-42c0-824a-7fd7dfdf077a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:11.228291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-71cd30c4-0b73-42c0-824a-7fd7dfdf077a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:11.241807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd30c4-0b73-42c0-824a-7fd7dfdf077a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:11.246406 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975baf24-8ae5-48c0-b932-066de5a60d44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:11.247061 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975baf24-8ae5-48c0-b932-066de5a60d44 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:11.247284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975baf24-8ae5-48c0-b932-066de5a60d44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:11.247552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975baf24-8ae5-48c0-b932-066de5a60d44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:11.254914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975baf24-8ae5-48c0-b932-066de5a60d44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:11.256332 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975baf24-8ae5-48c0-b932-066de5a60d44 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:11.266279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1379/2774] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:11.351384 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6001d-a86f-4e79-84e8-4d8556bdc5dc None None] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:11.351688 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6001d-a86f-4e79-84e8-4d8556bdc5dc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:11.352104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6001d-a86f-4e79-84e8-4d8556bdc5dc None None] PATH_INFO: `/` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:11.353646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1396/2775] 10.222.0.78 () {54 vars in 921 bytes} [Tue Aug 30 01:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:12:11.359403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ee7d5-4a94-4bc1-9181-de75a6930bb9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:11.359689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ee7d5-4a94-4bc1-9181-de75a6930bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:11.359960 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ee7d5-4a94-4bc1-9181-de75a6930bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:11.398528 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ea4ee7d5-4a94-4bc1-9181-de75a6930bb9 None None] MFA Rules not processed for user `f9b88c495fcb47a289a6a45276c891e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:11.399741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ea4ee7d5-4a94-4bc1-9181-de75a6930bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:11.467964 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1380/2776] 10.222.0.78 () {58 vars in 1018 bytes} [Tue Aug 30 01:12:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:11.478701 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6001d-a86f-4e79-84e8-4d8556bdc5dc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:11.489921 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6001d-a86f-4e79-84e8-4d8556bdc5dc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:11.493504 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca60a5-6671-423f-8f80-e4fd78b5c178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:11.494708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca60a5-6671-423f-8f80-e4fd78b5c178 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:11.495073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca60a5-6671-423f-8f80-e4fd78b5c178 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:11.495348 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca60a5-6671-423f-8f80-e4fd78b5c178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:11.505610 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca60a5-6671-423f-8f80-e4fd78b5c178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:11.506963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca60a5-6671-423f-8f80-e4fd78b5c178 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:11.518484 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1397/2777] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:11.582436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ee7d5-4a94-4bc1-9181-de75a6930bb9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:11.583800 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ea4ee7d5-4a94-4bc1-9181-de75a6930bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:11.651470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ee7d5-4a94-4bc1-9181-de75a6930bb9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:11.656513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:11.657125 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:11.657403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:11.657669 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:11.659363 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:11.671038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:11.672573 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:11.683046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1381/2778] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:13.850651 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca60a5-6671-423f-8f80-e4fd78b5c178 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:13.852175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e5ca60a5-6671-423f-8f80-e4fd78b5c178 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:13.868557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca60a5-6671-423f-8f80-e4fd78b5c178 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:13.874191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fd041-ee76-4914-8991-e9ba0a8af9c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:13.874803 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fd041-ee76-4914-8991-e9ba0a8af9c8 service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:13.875083 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fd041-ee76-4914-8991-e9ba0a8af9c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:13.875355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fd041-ee76-4914-8991-e9ba0a8af9c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:13.877140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9c4fd041-ee76-4914-8991-e9ba0a8af9c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:13.890174 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fd041-ee76-4914-8991-e9ba0a8af9c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:13.891751 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fd041-ee76-4914-8991-e9ba0a8af9c8 service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:13.903640 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1398/2779] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:17.551989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:17.553437 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:17.610920 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:17.630757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd72209-cd54-422f-9311-fdd632c3db9a service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:17.634395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168d22c-1177-4025-8822-8ea51b9b23e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:17.634991 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168d22c-1177-4025-8822-8ea51b9b23e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.635267 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168d22c-1177-4025-8822-8ea51b9b23e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.635533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168d22c-1177-4025-8822-8ea51b9b23e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:17.638179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4168d22c-1177-4025-8822-8ea51b9b23e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:17.696051 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168d22c-1177-4025-8822-8ea51b9b23e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:17.697492 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168d22c-1177-4025-8822-8ea51b9b23e2 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:17.707258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1382/2780] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:20.759824 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fd041-ee76-4914-8991-e9ba0a8af9c8 service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:20.784115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fd041-ee76-4914-8991-e9ba0a8af9c8 service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:20.795971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:20.796807 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:20.797328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:20.797785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:20.799767 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:20.814323 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:20.816519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:20.837055 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1399/2781] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:25.708221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168d22c-1177-4025-8822-8ea51b9b23e2 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:25.718305 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168d22c-1177-4025-8822-8ea51b9b23e2 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:25.721878 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c6981-ca98-4c86-8868-18c802370600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:25.722470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c6981-ca98-4c86-8868-18c802370600 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:25.722746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c6981-ca98-4c86-8868-18c802370600 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:25.723015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c6981-ca98-4c86-8868-18c802370600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:25.724741 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9d9c6981-ca98-4c86-8868-18c802370600 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:25.787844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c6981-ca98-4c86-8868-18c802370600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:25.789401 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c6981-ca98-4c86-8868-18c802370600 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:25.799683 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1383/2782] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:29.033290 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:29.034914 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:29.049319 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cfae2-ed6e-4177-9eb4-d350bcf0c36b service nova] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:29.054145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:29.054779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.055083 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.055529 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] PATH_INFO: `/v3/users/3ff0c4a8fd8d44bda56ceeb47cd49abf` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.056839 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff0c4a8fd8d44bda56ceeb47cd49abf)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:29.059568 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:29.106160 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff0c4a8fd8d44bda56ceeb47cd49abf'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.113156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff0c4a8fd8d44bda56ceeb47cd49abf'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.119925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff0c4a8fd8d44bda56ceeb47cd49abf'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.121152 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff0c4a8fd8d44bda56ceeb47cd49abf'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.133167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-a608b6ec-d350-4911-885a-61656fe78cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff0c4a8fd8d44bda56ceeb47cd49abf'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.141331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1400/2783] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:29 2022] DELETE /identity/v3/users/3ff0c4a8fd8d44bda56ceeb47cd49abf => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:29.316424 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c6981-ca98-4c86-8868-18c802370600 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:29.326858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c6981-ca98-4c86-8868-18c802370600 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:29.383722 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:29.384345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.384622 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.384893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] PATH_INFO: `/v3/projects/54768b167b1c4fefb901f640203e1c5d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.386252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54768b167b1c4fefb901f640203e1c5d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:29.388213 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:29.421833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:29.438943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54768b167b1c4fefb901f640203e1c5d', 'request_id': 'req-59c66b1c-1809-40da-98a0-65e75a9e98c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.446296 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54768b167b1c4fefb901f640203e1c5d', 'request_id': 'req-59c66b1c-1809-40da-98a0-65e75a9e98c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.453897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54768b167b1c4fefb901f640203e1c5d', 'request_id': 'req-59c66b1c-1809-40da-98a0-65e75a9e98c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.464972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] The token cache is being invalidate because project 54768b167b1c4fefb901f640203e1c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:29.465309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.465915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-59c66b1c-1809-40da-98a0-65e75a9e98c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:29.467394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1384/2784] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:29 2022] DELETE /identity/v3/projects/54768b167b1c4fefb901f640203e1c5d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:29.472312 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c243a2-96a2-47c6-87b1-b810621a427b None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.472597 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c243a2-96a2-47c6-87b1-b810621a427b None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.472867 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c243a2-96a2-47c6-87b1-b810621a427b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.493373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-48c243a2-96a2-47c6-87b1-b810621a427b None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:29.494565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-48c243a2-96a2-47c6-87b1-b810621a427b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:29.551973 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1401/2785] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:29.560233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec547-99ff-4c1e-973c-ad90d883fce0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.560451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec547-99ff-4c1e-973c-ad90d883fce0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.560718 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec547-99ff-4c1e-973c-ad90d883fce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.580778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d76ec547-99ff-4c1e-973c-ad90d883fce0 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:29.582035 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d76ec547-99ff-4c1e-973c-ad90d883fce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:29.590519 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1385/2786] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:29.594623 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c243a2-96a2-47c6-87b1-b810621a427b None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:29.604210 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c243a2-96a2-47c6-87b1-b810621a427b None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:29.608991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2646a87-21ce-4c15-bb13-1878f5aa2082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:29.609614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2646a87-21ce-4c15-bb13-1878f5aa2082 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.609896 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2646a87-21ce-4c15-bb13-1878f5aa2082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.610164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2646a87-21ce-4c15-bb13-1878f5aa2082 admin admin] PATH_INFO: `/v3/domains` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.610798 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2646a87-21ce-4c15-bb13-1878f5aa2082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81443) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:29.611096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2646a87-21ce-4c15-bb13-1878f5aa2082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:29.612708 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2646a87-21ce-4c15-bb13-1878f5aa2082 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:29.621410 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:29.624986 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1402/2787] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:12:29.629405 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec547-99ff-4c1e-973c-ad90d883fce0 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:29.639814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec547-99ff-4c1e-973c-ad90d883fce0 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:29.644586 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5064b-8f9f-4cbc-97e1-2a6f059ed25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:29.645219 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5064b-8f9f-4cbc-97e1-2a6f059ed25c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.645475 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5064b-8f9f-4cbc-97e1-2a6f059ed25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.645737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5064b-8f9f-4cbc-97e1-2a6f059ed25c admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.646631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5064b-8f9f-4cbc-97e1-2a6f059ed25c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:29.648618 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5064b-8f9f-4cbc-97e1-2a6f059ed25c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:29.685090 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1386/2788] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:29 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:29.689257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2646a87-21ce-4c15-bb13-1878f5aa2082 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:29.699969 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2646a87-21ce-4c15-bb13-1878f5aa2082 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:29.756761 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581aec9-beaa-4bbc-ac97-3104d9b5cb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:29.757388 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581aec9-beaa-4bbc-ac97-3104d9b5cb5c admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.757674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581aec9-beaa-4bbc-ac97-3104d9b5cb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.757945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581aec9-beaa-4bbc-ac97-3104d9b5cb5c admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.758574 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581aec9-beaa-4bbc-ac97-3104d9b5cb5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:29.760485 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581aec9-beaa-4bbc-ac97-3104d9b5cb5c admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:29.783396 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1403/2789] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:29 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:29.787569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5064b-8f9f-4cbc-97e1-2a6f059ed25c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:29.798922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5064b-8f9f-4cbc-97e1-2a6f059ed25c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:29.803628 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241333bf-79e4-4665-844e-7e3a9d077cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:29.804222 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241333bf-79e4-4665-844e-7e3a9d077cad admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.804504 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241333bf-79e4-4665-844e-7e3a9d077cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.804773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241333bf-79e4-4665-844e-7e3a9d077cad admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.805453 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241333bf-79e4-4665-844e-7e3a9d077cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:29.807028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241333bf-79e4-4665-844e-7e3a9d077cad admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:29.818059 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1387/2790] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:29.821887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581aec9-beaa-4bbc-ac97-3104d9b5cb5c admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:29.831676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581aec9-beaa-4bbc-ac97-3104d9b5cb5c admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:29.836226 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f58a53-6b1f-4cb9-921e-cb122325e09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:29.836808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f58a53-6b1f-4cb9-921e-cb122325e09a admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.837103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f58a53-6b1f-4cb9-921e-cb122325e09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.837375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f58a53-6b1f-4cb9-921e-cb122325e09a admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.837997 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f58a53-6b1f-4cb9-921e-cb122325e09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:29.839660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f58a53-6b1f-4cb9-921e-cb122325e09a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:29.849966 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1404/2791] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:29.854100 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241333bf-79e4-4665-844e-7e3a9d077cad admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:29.865177 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241333bf-79e4-4665-844e-7e3a9d077cad admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:29.870001 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56cb69-d281-46e5-aae0-85294c5495ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:29.871839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56cb69-d281-46e5-aae0-85294c5495ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.872118 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56cb69-d281-46e5-aae0-85294c5495ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.872393 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56cb69-d281-46e5-aae0-85294c5495ef admin admin] PATH_INFO: `/v3/projects/d19680d674bd4a8a8847483deacd69f1/users/b3abdb8bd2ed42c8a7a81ba48cdd68c8/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.888432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56cb69-d281-46e5-aae0-85294c5495ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19680d674bd4a8a8847483deacd69f1, user_id=b3abdb8bd2ed42c8a7a81ba48cdd68c8, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:29.891064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56cb69-d281-46e5-aae0-85294c5495ef admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:29.905601 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1388/2792] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:29 2022] PUT /identity/v3/projects/d19680d674bd4a8a8847483deacd69f1/users/b3abdb8bd2ed42c8a7a81ba48cdd68c8/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:29.918263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b36d-f373-43ab-8cae-168380146a44 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:29.918731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b36d-f373-43ab-8cae-168380146a44 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:29.919014 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b36d-f373-43ab-8cae-168380146a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:29.946339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-9a14b36d-f373-43ab-8cae-168380146a44 None None] MFA Rules not processed for user `b3abdb8bd2ed42c8a7a81ba48cdd68c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:29.947541 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-9a14b36d-f373-43ab-8cae-168380146a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:30.012606 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1405/2793] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:12:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:30.017482 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56cb69-d281-46e5-aae0-85294c5495ef admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:30.077653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56cb69-d281-46e5-aae0-85294c5495ef admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:30.082150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198868e-00e0-434e-8937-6c7e1c181a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:30.082737 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198868e-00e0-434e-8937-6c7e1c181a44 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:30.083011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198868e-00e0-434e-8937-6c7e1c181a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:30.083277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198868e-00e0-434e-8937-6c7e1c181a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:30.083897 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198868e-00e0-434e-8937-6c7e1c181a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:30.085797 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198868e-00e0-434e-8937-6c7e1c181a44 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:30.112582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1389/2794] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:30 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:30.116741 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14b36d-f373-43ab-8cae-168380146a44 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:30.127264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14b36d-f373-43ab-8cae-168380146a44 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:30.182646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616211c3-fd60-4d4f-875f-463f7f1ccca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:30.183286 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616211c3-fd60-4d4f-875f-463f7f1ccca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:30.183566 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616211c3-fd60-4d4f-875f-463f7f1ccca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:30.183834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616211c3-fd60-4d4f-875f-463f7f1ccca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:30.184470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616211c3-fd60-4d4f-875f-463f7f1ccca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:30.186416 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616211c3-fd60-4d4f-875f-463f7f1ccca9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:30.201359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:30.210441 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1406/2795] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:30 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:30.211600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198868e-00e0-434e-8937-6c7e1c181a44 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:30.222552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198868e-00e0-434e-8937-6c7e1c181a44 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:30.227179 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034b042-a9f8-40e9-86fd-ffac6a39b883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:30.227762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034b042-a9f8-40e9-86fd-ffac6a39b883 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:30.228038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034b042-a9f8-40e9-86fd-ffac6a39b883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:30.228304 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034b042-a9f8-40e9-86fd-ffac6a39b883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:30.228931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034b042-a9f8-40e9-86fd-ffac6a39b883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:30.231387 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034b042-a9f8-40e9-86fd-ffac6a39b883 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:30.241782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1390/2796] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:30.246180 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616211c3-fd60-4d4f-875f-463f7f1ccca9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:30.256198 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616211c3-fd60-4d4f-875f-463f7f1ccca9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:30.261813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd80aa-3c02-4a54-9651-f89ea1af210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:30.262796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd80aa-3c02-4a54-9651-f89ea1af210e admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:30.263101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd80aa-3c02-4a54-9651-f89ea1af210e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:30.263375 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd80aa-3c02-4a54-9651-f89ea1af210e admin admin] PATH_INFO: `/v3/projects/c055f56700404b49bdd6cf80d0666794/users/478d0f53c45649899ee06c8259c7e101/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:30.279540 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd80aa-3c02-4a54-9651-f89ea1af210e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c055f56700404b49bdd6cf80d0666794, user_id=478d0f53c45649899ee06c8259c7e101, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:30.282238 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd80aa-3c02-4a54-9651-f89ea1af210e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:30.294766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1407/2797] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:30 2022] PUT /identity/v3/projects/c055f56700404b49bdd6cf80d0666794/users/478d0f53c45649899ee06c8259c7e101/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:30.298757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034b042-a9f8-40e9-86fd-ffac6a39b883 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:30.308879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034b042-a9f8-40e9-86fd-ffac6a39b883 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:30.319924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:30.365955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077a14b-1392-433c-8a41-6d47646e57be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:30.366469 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077a14b-1392-433c-8a41-6d47646e57be admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:30.366742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077a14b-1392-433c-8a41-6d47646e57be admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:30.367009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077a14b-1392-433c-8a41-6d47646e57be admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:30.367632 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077a14b-1392-433c-8a41-6d47646e57be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:30.369228 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077a14b-1392-433c-8a41-6d47646e57be admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:30.379977 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1391/2798] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:30.384400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd80aa-3c02-4a54-9651-f89ea1af210e admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:30.395658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd80aa-3c02-4a54-9651-f89ea1af210e admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:30.400564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b489db7-5861-4cec-a214-377b28d35051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:30.401239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b489db7-5861-4cec-a214-377b28d35051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:30.401513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b489db7-5861-4cec-a214-377b28d35051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:30.401777 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b489db7-5861-4cec-a214-377b28d35051 admin admin] PATH_INFO: `/v3/projects/c055f56700404b49bdd6cf80d0666794/users/478d0f53c45649899ee06c8259c7e101/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:30.404503 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b489db7-5861-4cec-a214-377b28d35051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c055f56700404b49bdd6cf80d0666794, user_id=478d0f53c45649899ee06c8259c7e101, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:30.407170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b489db7-5861-4cec-a214-377b28d35051 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:30.419346 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1408/2799] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:30 2022] PUT /identity/v3/projects/c055f56700404b49bdd6cf80d0666794/users/478d0f53c45649899ee06c8259c7e101/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:30.429672 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8ee7d-59a1-47ce-b4b5-aacf084885da None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:30.429959 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8ee7d-59a1-47ce-b4b5-aacf084885da None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:30.430233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8ee7d-59a1-47ce-b4b5-aacf084885da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:30.458038 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cae8ee7d-59a1-47ce-b4b5-aacf084885da None None] MFA Rules not processed for user `478d0f53c45649899ee06c8259c7e101`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:30.459309 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cae8ee7d-59a1-47ce-b4b5-aacf084885da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:30.529708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1392/2800] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:12:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:30.540933 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b489db7-5861-4cec-a214-377b28d35051 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:30.542703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3b489db7-5861-4cec-a214-377b28d35051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:30.600891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b489db7-5861-4cec-a214-377b28d35051 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:30.604591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a3799-9dce-4e4a-8ca7-99f979ed8f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:30.605334 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a3799-9dce-4e4a-8ca7-99f979ed8f69 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:30.605625 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a3799-9dce-4e4a-8ca7-99f979ed8f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:30.605899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a3799-9dce-4e4a-8ca7-99f979ed8f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:30.614689 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a3799-9dce-4e4a-8ca7-99f979ed8f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:30.616164 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a3799-9dce-4e4a-8ca7-99f979ed8f69 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:30.625700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1409/2801] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:31.016735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8ee7d-59a1-47ce-b4b5-aacf084885da None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:31.084545 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8ee7d-59a1-47ce-b4b5-aacf084885da None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:31.089650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:31.090487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:31.090763 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:31.091034 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] PATH_INFO: `/v3/users/b3abdb8bd2ed42c8a7a81ba48cdd68c8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:31.092169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3abdb8bd2ed42c8a7a81ba48cdd68c8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:31.094167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:31.134749 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:31.142186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3abdb8bd2ed42c8a7a81ba48cdd68c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.149556 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3abdb8bd2ed42c8a7a81ba48cdd68c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.150173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3abdb8bd2ed42c8a7a81ba48cdd68c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.150793 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3abdb8bd2ed42c8a7a81ba48cdd68c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.162687 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3abdb8bd2ed42c8a7a81ba48cdd68c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.175913 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1393/2802] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:31 2022] DELETE /identity/v3/users/b3abdb8bd2ed42c8a7a81ba48cdd68c8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:31.180421 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101df92-b76e-47f2-9219-40418e6ad798 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:31.180906 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101df92-b76e-47f2-9219-40418e6ad798 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:31.181006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101df92-b76e-47f2-9219-40418e6ad798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:31.200148 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-8101df92-b76e-47f2-9219-40418e6ad798 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:31.201372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8101df92-b76e-47f2-9219-40418e6ad798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:31.259761 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1410/2803] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:31.269746 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:31.271439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:31.328350 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d2c57-6de0-4bfa-94d0-20d6cc89e254 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:31.332054 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f34b3-c1de-4b33-b880-ec576766fde3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:31.332639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f34b3-c1de-4b33-b880-ec576766fde3 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:31.332914 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f34b3-c1de-4b33-b880-ec576766fde3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:31.333215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f34b3-c1de-4b33-b880-ec576766fde3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:31.342291 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f34b3-c1de-4b33-b880-ec576766fde3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:31.343686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f34b3-c1de-4b33-b880-ec576766fde3 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:31.353208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1394/2804] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:12:31.620558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101df92-b76e-47f2-9219-40418e6ad798 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:31.622494 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-8101df92-b76e-47f2-9219-40418e6ad798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:31.636058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101df92-b76e-47f2-9219-40418e6ad798 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:31.641744 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:31.642442 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:31.642654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:31.642925 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] PATH_INFO: `/v3/projects/d19680d674bd4a8a8847483deacd69f1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:31.644060 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19680d674bd4a8a8847483deacd69f1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:31.646269 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:31.694218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19680d674bd4a8a8847483deacd69f1', 'request_id': 'req-f42586b6-b473-46a8-9b38-98f559051ebd'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.700366 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19680d674bd4a8a8847483deacd69f1', 'request_id': 'req-f42586b6-b473-46a8-9b38-98f559051ebd'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.704159 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:31.707513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19680d674bd4a8a8847483deacd69f1', 'request_id': 'req-f42586b6-b473-46a8-9b38-98f559051ebd'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.719342 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] The token cache is being invalidate because project d19680d674bd4a8a8847483deacd69f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:31.719630 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.720268 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.721784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1411/2805] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:31 2022] DELETE /identity/v3/projects/d19680d674bd4a8a8847483deacd69f1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:31.725757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f34b3-c1de-4b33-b880-ec576766fde3 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:31.736141 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f34b3-c1de-4b33-b880-ec576766fde3 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:31.795796 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:31.796420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:31.796699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:31.796972 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] PATH_INFO: `/v3/users/478d0f53c45649899ee06c8259c7e101` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:31.798121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=478d0f53c45649899ee06c8259c7e101)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:31.800024 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:31.847276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478d0f53c45649899ee06c8259c7e101'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.856371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478d0f53c45649899ee06c8259c7e101'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.857007 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478d0f53c45649899ee06c8259c7e101'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.857627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478d0f53c45649899ee06c8259c7e101'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.862766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:31.870158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-72d1e064-e6f4-4faa-b978-a74a80703df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478d0f53c45649899ee06c8259c7e101'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:31.882323 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1395/2806] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:31 2022] DELETE /identity/v3/users/478d0f53c45649899ee06c8259c7e101 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:32.146558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:32.147982 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:32.161564 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42586b6-b473-46a8-9b38-98f559051ebd admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:32.218230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:32.218875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.219108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.219373 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] PATH_INFO: `/v3/projects/c055f56700404b49bdd6cf80d0666794` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.220474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c055f56700404b49bdd6cf80d0666794)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:32.222399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:32.268759 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c055f56700404b49bdd6cf80d0666794', 'request_id': 'req-731a89d0-017f-4bc3-b981-d26ccf2f404e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:32.275355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c055f56700404b49bdd6cf80d0666794', 'request_id': 'req-731a89d0-017f-4bc3-b981-d26ccf2f404e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:32.282167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c055f56700404b49bdd6cf80d0666794', 'request_id': 'req-731a89d0-017f-4bc3-b981-d26ccf2f404e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:32.292581 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] The token cache is being invalidate because project c055f56700404b49bdd6cf80d0666794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:32.293379 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:32.294006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-731a89d0-017f-4bc3-b981-d26ccf2f404e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:32.295496 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1412/2807] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:32 2022] DELETE /identity/v3/projects/c055f56700404b49bdd6cf80d0666794 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:32.300421 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edba7d-8282-45d5-8841-358563932eb0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.300705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edba7d-8282-45d5-8841-358563932eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.300975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edba7d-8282-45d5-8841-358563932eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.321165 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-73edba7d-8282-45d5-8841-358563932eb0 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:32.322361 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-73edba7d-8282-45d5-8841-358563932eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:32.381987 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1396/2808] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:32.391204 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb76be-afe7-4148-bc33-d7b44449d9f6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.391542 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb76be-afe7-4148-bc33-d7b44449d9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.391813 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb76be-afe7-4148-bc33-d7b44449d9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.412103 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-0ffb76be-afe7-4148-bc33-d7b44449d9f6 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:32.413338 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-0ffb76be-afe7-4148-bc33-d7b44449d9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:32.422027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1413/2809] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:32.426303 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edba7d-8282-45d5-8841-358563932eb0 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:32.437226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edba7d-8282-45d5-8841-358563932eb0 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:32.441837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca20c15-e2d6-46b3-9361-c4424a1284b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:32.442429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca20c15-e2d6-46b3-9361-c4424a1284b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.442743 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca20c15-e2d6-46b3-9361-c4424a1284b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.443005 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca20c15-e2d6-46b3-9361-c4424a1284b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.443649 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca20c15-e2d6-46b3-9361-c4424a1284b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:32.443947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca20c15-e2d6-46b3-9361-c4424a1284b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:32.445568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca20c15-e2d6-46b3-9361-c4424a1284b0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:32.458278 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1397/2810] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:12:32.462709 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb76be-afe7-4148-bc33-d7b44449d9f6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:32.472324 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb76be-afe7-4148-bc33-d7b44449d9f6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:32.476972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c5e67-fc1e-45d9-a047-7dde6c405f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:32.477594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5e67-fc1e-45d9-a047-7dde6c405f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.477870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5e67-fc1e-45d9-a047-7dde6c405f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.478137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5e67-fc1e-45d9-a047-7dde6c405f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.478780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c5e67-fc1e-45d9-a047-7dde6c405f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:32.480742 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c5e67-fc1e-45d9-a047-7dde6c405f82 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:32.510987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1414/2811] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:32 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:32.516156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca20c15-e2d6-46b3-9361-c4424a1284b0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:32.526460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca20c15-e2d6-46b3-9361-c4424a1284b0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:32.583644 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad79ab-5bb5-47c1-a349-abb0ac74b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:32.584238 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad79ab-5bb5-47c1-a349-abb0ac74b7ed admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.584514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad79ab-5bb5-47c1-a349-abb0ac74b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.584785 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad79ab-5bb5-47c1-a349-abb0ac74b7ed admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.585446 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad79ab-5bb5-47c1-a349-abb0ac74b7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:32.587339 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad79ab-5bb5-47c1-a349-abb0ac74b7ed admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:32.607946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1398/2812] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:32 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:32.612042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c5e67-fc1e-45d9-a047-7dde6c405f82 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:32.623012 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c5e67-fc1e-45d9-a047-7dde6c405f82 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:32.627809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40653093-4d3c-4497-8445-32a256126eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:32.628407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40653093-4d3c-4497-8445-32a256126eba admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.628707 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40653093-4d3c-4497-8445-32a256126eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.628999 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40653093-4d3c-4497-8445-32a256126eba admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.629658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40653093-4d3c-4497-8445-32a256126eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:32.631230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40653093-4d3c-4497-8445-32a256126eba admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:32.641417 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1415/2813] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:32.645210 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad79ab-5bb5-47c1-a349-abb0ac74b7ed admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:32.655731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad79ab-5bb5-47c1-a349-abb0ac74b7ed admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:32.660297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfa37a-5aef-4477-bbae-aec8194b5b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:32.660879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfa37a-5aef-4477-bbae-aec8194b5b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.661186 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfa37a-5aef-4477-bbae-aec8194b5b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.661461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfa37a-5aef-4477-bbae-aec8194b5b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.662121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfa37a-5aef-4477-bbae-aec8194b5b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:32.663696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfa37a-5aef-4477-bbae-aec8194b5b36 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:32.675025 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1399/2814] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:32.679035 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40653093-4d3c-4497-8445-32a256126eba admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:32.689580 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40653093-4d3c-4497-8445-32a256126eba admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:32.694146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db60763-920a-4f10-8824-0efb6c813586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:32.694780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db60763-920a-4f10-8824-0efb6c813586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.695058 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db60763-920a-4f10-8824-0efb6c813586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.695328 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db60763-920a-4f10-8824-0efb6c813586 admin admin] PATH_INFO: `/v3/projects/0bb62216b46a40fd9fd40b0edc62af57/users/bdeb9d6c692d47cf9d845a8504235905/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.709769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db60763-920a-4f10-8824-0efb6c813586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb62216b46a40fd9fd40b0edc62af57, user_id=bdeb9d6c692d47cf9d845a8504235905, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:32.712414 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db60763-920a-4f10-8824-0efb6c813586 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:32.724891 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1416/2815] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:32 2022] PUT /identity/v3/projects/0bb62216b46a40fd9fd40b0edc62af57/users/bdeb9d6c692d47cf9d845a8504235905/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:32.736961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79486e7-a1b8-41b6-8595-3031f4dfa1a1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.737297 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79486e7-a1b8-41b6-8595-3031f4dfa1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.737572 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79486e7-a1b8-41b6-8595-3031f4dfa1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.766373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b79486e7-a1b8-41b6-8595-3031f4dfa1a1 None None] MFA Rules not processed for user `bdeb9d6c692d47cf9d845a8504235905`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:32.767513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b79486e7-a1b8-41b6-8595-3031f4dfa1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:32.800470 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:32.834710 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1400/2816] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:12:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:32.839359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db60763-920a-4f10-8824-0efb6c813586 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:32.852779 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:32.903791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db60763-920a-4f10-8824-0efb6c813586 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:32.908359 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb551b3-8dcf-4649-8143-b96c554508e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:32.908953 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb551b3-8dcf-4649-8143-b96c554508e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:32.909263 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb551b3-8dcf-4649-8143-b96c554508e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:32.909528 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb551b3-8dcf-4649-8143-b96c554508e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:32.910154 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb551b3-8dcf-4649-8143-b96c554508e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:32.912036 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb551b3-8dcf-4649-8143-b96c554508e0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:32.938086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1417/2817] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:32 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:32.942221 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79486e7-a1b8-41b6-8595-3031f4dfa1a1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:32.952903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79486e7-a1b8-41b6-8595-3031f4dfa1a1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:33.007705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c482328-623d-411d-8d5c-b76bbf7d06fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:33.008282 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c482328-623d-411d-8d5c-b76bbf7d06fa admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:33.008560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c482328-623d-411d-8d5c-b76bbf7d06fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:33.008825 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c482328-623d-411d-8d5c-b76bbf7d06fa admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:33.009488 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c482328-623d-411d-8d5c-b76bbf7d06fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:33.011407 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c482328-623d-411d-8d5c-b76bbf7d06fa admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:33.032922 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1401/2818] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:32 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:33.036868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb551b3-8dcf-4649-8143-b96c554508e0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:33.048420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb551b3-8dcf-4649-8143-b96c554508e0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:33.053917 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e724188-eac4-417c-adec-b9e4508006b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:33.054509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e724188-eac4-417c-adec-b9e4508006b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:33.054789 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e724188-eac4-417c-adec-b9e4508006b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:33.055059 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e724188-eac4-417c-adec-b9e4508006b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:33.055703 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e724188-eac4-417c-adec-b9e4508006b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:33.057330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e724188-eac4-417c-adec-b9e4508006b5 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:33.068190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1418/2819] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:33.072335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c482328-623d-411d-8d5c-b76bbf7d06fa admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:33.084685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c482328-623d-411d-8d5c-b76bbf7d06fa admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:33.091765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69dc26-4553-4e7b-af28-93241eb09cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:33.092414 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69dc26-4553-4e7b-af28-93241eb09cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:33.092696 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69dc26-4553-4e7b-af28-93241eb09cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:33.092970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69dc26-4553-4e7b-af28-93241eb09cfd admin admin] PATH_INFO: `/v3/projects/2ca1a043b3f241a89a5318e16d7a4800/users/126075a0ab764430b30c1de79cfb7cc5/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:33.108690 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69dc26-4553-4e7b-af28-93241eb09cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca1a043b3f241a89a5318e16d7a4800, user_id=126075a0ab764430b30c1de79cfb7cc5, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:33.111392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69dc26-4553-4e7b-af28-93241eb09cfd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:33.124154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1402/2820] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:33 2022] PUT /identity/v3/projects/2ca1a043b3f241a89a5318e16d7a4800/users/126075a0ab764430b30c1de79cfb7cc5/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:33.128146 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e724188-eac4-417c-adec-b9e4508006b5 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:33.138230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e724188-eac4-417c-adec-b9e4508006b5 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:33.195424 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567f2da-b45c-4955-9a7a-a5fd729db8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:33.196009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567f2da-b45c-4955-9a7a-a5fd729db8ea admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:33.196283 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567f2da-b45c-4955-9a7a-a5fd729db8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:33.196549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567f2da-b45c-4955-9a7a-a5fd729db8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:33.197220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567f2da-b45c-4955-9a7a-a5fd729db8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:33.198797 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567f2da-b45c-4955-9a7a-a5fd729db8ea admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:33.208418 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1419/2821] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:33.212703 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69dc26-4553-4e7b-af28-93241eb09cfd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:33.223650 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69dc26-4553-4e7b-af28-93241eb09cfd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:33.228320 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4e109-cd6e-4261-9f48-dac634df1566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:33.228954 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4e109-cd6e-4261-9f48-dac634df1566 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:33.229258 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4e109-cd6e-4261-9f48-dac634df1566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:33.229530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4e109-cd6e-4261-9f48-dac634df1566 admin admin] PATH_INFO: `/v3/projects/2ca1a043b3f241a89a5318e16d7a4800/users/126075a0ab764430b30c1de79cfb7cc5/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:33.232198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4e109-cd6e-4261-9f48-dac634df1566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca1a043b3f241a89a5318e16d7a4800, user_id=126075a0ab764430b30c1de79cfb7cc5, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:33.234828 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4e109-cd6e-4261-9f48-dac634df1566 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:33.248176 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1403/2822] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:33 2022] PUT /identity/v3/projects/2ca1a043b3f241a89a5318e16d7a4800/users/126075a0ab764430b30c1de79cfb7cc5/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:33.258595 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e07bb8-6e6f-400b-ab10-e970317497d6 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:33.259100 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e07bb8-6e6f-400b-ab10-e970317497d6 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:33.259351 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e07bb8-6e6f-400b-ab10-e970317497d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:33.289873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-f0e07bb8-6e6f-400b-ab10-e970317497d6 None None] MFA Rules not processed for user `126075a0ab764430b30c1de79cfb7cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:33.291008 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-f0e07bb8-6e6f-400b-ab10-e970317497d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:33.357262 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1420/2823] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:12:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:33.368168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4e109-cd6e-4261-9f48-dac634df1566 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:33.369962 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e2e4e109-cd6e-4261-9f48-dac634df1566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:33.426254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4e109-cd6e-4261-9f48-dac634df1566 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:33.429898 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a6211-1627-4895-887c-44164e2a95c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:33.430483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a6211-1627-4895-887c-44164e2a95c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:33.430757 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a6211-1627-4895-887c-44164e2a95c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:33.431044 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a6211-1627-4895-887c-44164e2a95c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:33.471191 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:33.487777 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a6211-1627-4895-887c-44164e2a95c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:33.489215 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a6211-1627-4895-887c-44164e2a95c1 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:33.497892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1404/2824] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:38.108717 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e07bb8-6e6f-400b-ab10-e970317497d6 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:38.118200 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e07bb8-6e6f-400b-ab10-e970317497d6 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:38.122009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51927944-bc3e-4c44-a909-38f6fe2c6231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:38.122598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51927944-bc3e-4c44-a909-38f6fe2c6231 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:38.122870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51927944-bc3e-4c44-a909-38f6fe2c6231 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:38.123134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51927944-bc3e-4c44-a909-38f6fe2c6231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:38.129855 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:38.132387 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51927944-bc3e-4c44-a909-38f6fe2c6231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:38.134278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51927944-bc3e-4c44-a909-38f6fe2c6231 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:38.145195 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1421/2825] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:43.493867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=9 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:48.140951 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:02.978298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a6211-1627-4895-887c-44164e2a95c1 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:03.039975 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a6211-1627-4895-887c-44164e2a95c1 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:03.045008 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:03.045657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:03.045933 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:03.046202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] PATH_INFO: `/v3/users/bdeb9d6c692d47cf9d845a8504235905` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:03.047315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdeb9d6c692d47cf9d845a8504235905)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:03.049249 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:03.094194 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdeb9d6c692d47cf9d845a8504235905'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.101147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdeb9d6c692d47cf9d845a8504235905'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.101812 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdeb9d6c692d47cf9d845a8504235905'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.102427 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdeb9d6c692d47cf9d845a8504235905'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.115365 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdeb9d6c692d47cf9d845a8504235905'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.127429 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1405/2826] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:02 2022] DELETE /identity/v3/users/bdeb9d6c692d47cf9d845a8504235905 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:03.131924 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395ce82-4f1b-46ce-ad88-ea41d425e587 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:03.132434 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395ce82-4f1b-46ce-ad88-ea41d425e587 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:03.132552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395ce82-4f1b-46ce-ad88-ea41d425e587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:03.151524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3395ce82-4f1b-46ce-ad88-ea41d425e587 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:03.153557 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3395ce82-4f1b-46ce-ad88-ea41d425e587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:03.210784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1422/2827] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:03.221045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:03.222514 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:03.281451 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597f5c7-159b-48f1-a6cb-871ffe2d87a6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:03.285147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec926c-f00e-4005-b81e-abc44fc6fab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:03.285744 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec926c-f00e-4005-b81e-abc44fc6fab9 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:03.286016 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec926c-f00e-4005-b81e-abc44fc6fab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:03.286279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec926c-f00e-4005-b81e-abc44fc6fab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:03.295394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec926c-f00e-4005-b81e-abc44fc6fab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:03.296788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec926c-f00e-4005-b81e-abc44fc6fab9 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:03.305970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1406/2828] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:03.574163 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395ce82-4f1b-46ce-ad88-ea41d425e587 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:03.576189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3395ce82-4f1b-46ce-ad88-ea41d425e587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:03.590856 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395ce82-4f1b-46ce-ad88-ea41d425e587 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:03.595668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:03.596299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:03.596571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:03.596837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] PATH_INFO: `/v3/projects/0bb62216b46a40fd9fd40b0edc62af57` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:03.598004 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bb62216b46a40fd9fd40b0edc62af57)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:03.599904 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:03.645746 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bb62216b46a40fd9fd40b0edc62af57', 'request_id': 'req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.651927 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bb62216b46a40fd9fd40b0edc62af57', 'request_id': 'req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.659769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bb62216b46a40fd9fd40b0edc62af57', 'request_id': 'req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.670674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] The token cache is being invalidate because project 0bb62216b46a40fd9fd40b0edc62af57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:03.670963 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.671586 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.673115 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1423/2829] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:03 2022] DELETE /identity/v3/projects/0bb62216b46a40fd9fd40b0edc62af57 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:03.677144 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec926c-f00e-4005-b81e-abc44fc6fab9 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:03.687395 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec926c-f00e-4005-b81e-abc44fc6fab9 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:03.705801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:03.747895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:03.748439 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:03.748720 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:03.748993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] PATH_INFO: `/v3/users/126075a0ab764430b30c1de79cfb7cc5` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:03.750150 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126075a0ab764430b30c1de79cfb7cc5)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:03.752056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:03.798015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126075a0ab764430b30c1de79cfb7cc5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.806501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126075a0ab764430b30c1de79cfb7cc5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.807175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126075a0ab764430b30c1de79cfb7cc5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.807781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126075a0ab764430b30c1de79cfb7cc5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.821028 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-50778b2e-33bd-4f01-94a4-c7e7061802d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126075a0ab764430b30c1de79cfb7cc5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:03.833893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1407/2830] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:03 2022] DELETE /identity/v3/users/126075a0ab764430b30c1de79cfb7cc5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:04.100826 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:04.102405 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:04.119112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc608c9-a2fe-44d9-89bf-de5592a7d4ca admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:04.176477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:04.177158 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.177383 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.177646 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] PATH_INFO: `/v3/projects/2ca1a043b3f241a89a5318e16d7a4800` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.178735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca1a043b3f241a89a5318e16d7a4800)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:04.180633 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:04.210692 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:04.228043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca1a043b3f241a89a5318e16d7a4800', 'request_id': 'req-d454adfa-3dca-4685-9904-5318c5ad8332'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:04.234239 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca1a043b3f241a89a5318e16d7a4800', 'request_id': 'req-d454adfa-3dca-4685-9904-5318c5ad8332'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:04.242940 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca1a043b3f241a89a5318e16d7a4800', 'request_id': 'req-d454adfa-3dca-4685-9904-5318c5ad8332'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:04.254299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] The token cache is being invalidate because project 2ca1a043b3f241a89a5318e16d7a4800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:04.254590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:04.255230 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-d454adfa-3dca-4685-9904-5318c5ad8332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:04.256730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1424/2831] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:04 2022] DELETE /identity/v3/projects/2ca1a043b3f241a89a5318e16d7a4800 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:04.261592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55de57-ee9f-492e-9ebf-73314146bcba None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.263127 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55de57-ee9f-492e-9ebf-73314146bcba None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.263398 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55de57-ee9f-492e-9ebf-73314146bcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.283698 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4a55de57-ee9f-492e-9ebf-73314146bcba None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:04.284816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4a55de57-ee9f-492e-9ebf-73314146bcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:04.343404 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1408/2832] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:04.406590 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb54b1-2912-45b2-aef2-a77e3ffb0afc None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.406872 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb54b1-2912-45b2-aef2-a77e3ffb0afc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.407145 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb54b1-2912-45b2-aef2-a77e3ffb0afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.426861 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-3acb54b1-2912-45b2-aef2-a77e3ffb0afc None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:04.428061 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-3acb54b1-2912-45b2-aef2-a77e3ffb0afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:04.436600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1425/2833] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:13:04.440736 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55de57-ee9f-492e-9ebf-73314146bcba None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:04.450989 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55de57-ee9f-492e-9ebf-73314146bcba None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:04.455524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941b61-997d-4b95-89af-9b03c4dc2f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:04.456113 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941b61-997d-4b95-89af-9b03c4dc2f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.456388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941b61-997d-4b95-89af-9b03c4dc2f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.456657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941b61-997d-4b95-89af-9b03c4dc2f47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.457326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f941b61-997d-4b95-89af-9b03c4dc2f47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:13:04.457629 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f941b61-997d-4b95-89af-9b03c4dc2f47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:04.459198 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f941b61-997d-4b95-89af-9b03c4dc2f47 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:04.470931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1409/2834] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:13:04.475274 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb54b1-2912-45b2-aef2-a77e3ffb0afc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:04.484656 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb54b1-2912-45b2-aef2-a77e3ffb0afc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:04.489308 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c90a4-05dd-437f-b4ba-b7a81b98b637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:04.489895 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c90a4-05dd-437f-b4ba-b7a81b98b637 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.490175 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c90a4-05dd-437f-b4ba-b7a81b98b637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.490448 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c90a4-05dd-437f-b4ba-b7a81b98b637 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.491083 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c90a4-05dd-437f-b4ba-b7a81b98b637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:04.492968 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c90a4-05dd-437f-b4ba-b7a81b98b637 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:04.520532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1426/2835] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:04 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:04.525102 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941b61-997d-4b95-89af-9b03c4dc2f47 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:04.535193 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941b61-997d-4b95-89af-9b03c4dc2f47 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:04.561373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:04.593174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d4ae7-8296-4037-9886-d354912c8a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:04.593773 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d4ae7-8296-4037-9886-d354912c8a36 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.594053 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d4ae7-8296-4037-9886-d354912c8a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.594326 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d4ae7-8296-4037-9886-d354912c8a36 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.594950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d4ae7-8296-4037-9886-d354912c8a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:04.596838 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d4ae7-8296-4037-9886-d354912c8a36 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:04.617631 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1410/2836] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:04 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:04.622690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c90a4-05dd-437f-b4ba-b7a81b98b637 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:04.632931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c90a4-05dd-437f-b4ba-b7a81b98b637 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:04.638355 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120feff-9308-4f6b-ba74-4db010912059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:04.638945 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120feff-9308-4f6b-ba74-4db010912059 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.639221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120feff-9308-4f6b-ba74-4db010912059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.639487 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120feff-9308-4f6b-ba74-4db010912059 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.640116 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120feff-9308-4f6b-ba74-4db010912059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:04.641800 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120feff-9308-4f6b-ba74-4db010912059 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:04.651897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1427/2837] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:04.655864 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d4ae7-8296-4037-9886-d354912c8a36 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:04.665731 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d4ae7-8296-4037-9886-d354912c8a36 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:04.671000 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3be782-4353-4923-8f13-e9dd519d7874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:04.671588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3be782-4353-4923-8f13-e9dd519d7874 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.671867 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3be782-4353-4923-8f13-e9dd519d7874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.672139 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3be782-4353-4923-8f13-e9dd519d7874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.672778 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3be782-4353-4923-8f13-e9dd519d7874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:04.674393 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3be782-4353-4923-8f13-e9dd519d7874 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:04.685653 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1411/2838] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:04.689676 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120feff-9308-4f6b-ba74-4db010912059 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:04.700157 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120feff-9308-4f6b-ba74-4db010912059 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:04.704814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4de51-8be6-4bed-b2ab-77491a65fbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:04.705500 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4de51-8be6-4bed-b2ab-77491a65fbce admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.705775 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4de51-8be6-4bed-b2ab-77491a65fbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.706043 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4de51-8be6-4bed-b2ab-77491a65fbce admin admin] PATH_INFO: `/v3/projects/8a8fdea8cdf64e47b3897bacb2300986/users/405b88aaaedf441ba5cff4dec8f0d1c8/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.720773 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4de51-8be6-4bed-b2ab-77491a65fbce admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8fdea8cdf64e47b3897bacb2300986, user_id=405b88aaaedf441ba5cff4dec8f0d1c8, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:04.723499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4de51-8be6-4bed-b2ab-77491a65fbce admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:04.735868 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1428/2839] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:04 2022] PUT /identity/v3/projects/8a8fdea8cdf64e47b3897bacb2300986/users/405b88aaaedf441ba5cff4dec8f0d1c8/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:04.747532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62746dec-41d6-46d2-b496-760d2c29aaa3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.747849 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62746dec-41d6-46d2-b496-760d2c29aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.748123 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62746dec-41d6-46d2-b496-760d2c29aaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.776499 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-62746dec-41d6-46d2-b496-760d2c29aaa3 None None] MFA Rules not processed for user `405b88aaaedf441ba5cff4dec8f0d1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:04.777903 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-62746dec-41d6-46d2-b496-760d2c29aaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:04.844826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1412/2840] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:04.849231 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4de51-8be6-4bed-b2ab-77491a65fbce admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:04.910361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4de51-8be6-4bed-b2ab-77491a65fbce admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:04.915069 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59119d-857b-4aa0-8f55-b7da6b01e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:04.915658 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59119d-857b-4aa0-8f55-b7da6b01e259 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:04.915931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59119d-857b-4aa0-8f55-b7da6b01e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:04.916196 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59119d-857b-4aa0-8f55-b7da6b01e259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:04.916821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59119d-857b-4aa0-8f55-b7da6b01e259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:04.919406 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59119d-857b-4aa0-8f55-b7da6b01e259 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:04.945299 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1429/2841] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:04 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:04.949512 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62746dec-41d6-46d2-b496-760d2c29aaa3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:04.960233 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62746dec-41d6-46d2-b496-760d2c29aaa3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:05.022565 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4390119-8f28-467a-a078-0d3af639762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:05.023168 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4390119-8f28-467a-a078-0d3af639762d admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:05.023445 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4390119-8f28-467a-a078-0d3af639762d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:05.023713 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4390119-8f28-467a-a078-0d3af639762d admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:05.024328 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4390119-8f28-467a-a078-0d3af639762d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:05.026274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4390119-8f28-467a-a078-0d3af639762d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:05.047096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1413/2842] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:04 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:05.051362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59119d-857b-4aa0-8f55-b7da6b01e259 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:05.061559 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59119d-857b-4aa0-8f55-b7da6b01e259 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:05.066090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c7008-7f48-4420-9391-529ba200b305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:05.066674 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c7008-7f48-4420-9391-529ba200b305 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:05.066949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c7008-7f48-4420-9391-529ba200b305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:05.067211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c7008-7f48-4420-9391-529ba200b305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:05.067833 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c7008-7f48-4420-9391-529ba200b305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:05.069451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c7008-7f48-4420-9391-529ba200b305 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:05.080432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1430/2843] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:05.084479 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4390119-8f28-467a-a078-0d3af639762d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:05.094549 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4390119-8f28-467a-a078-0d3af639762d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:05.099224 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e295e-36e7-4890-aacf-994710cb7285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:05.099795 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e295e-36e7-4890-aacf-994710cb7285 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:05.100072 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e295e-36e7-4890-aacf-994710cb7285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:05.100342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e295e-36e7-4890-aacf-994710cb7285 admin admin] PATH_INFO: `/v3/projects/e36291540e9348dd88350d1c550dd0b9/users/30a6a750770847a2a2f9a0842bca24ef/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:05.115513 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e295e-36e7-4890-aacf-994710cb7285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e36291540e9348dd88350d1c550dd0b9, user_id=30a6a750770847a2a2f9a0842bca24ef, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:05.118275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e295e-36e7-4890-aacf-994710cb7285 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:05.134774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1414/2844] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:05 2022] PUT /identity/v3/projects/e36291540e9348dd88350d1c550dd0b9/users/30a6a750770847a2a2f9a0842bca24ef/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:05.139288 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c7008-7f48-4420-9391-529ba200b305 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:05.150184 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c7008-7f48-4420-9391-529ba200b305 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:05.171622 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:05.211536 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825350e0-f634-4c5b-8304-a5ae53a3621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:05.212330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825350e0-f634-4c5b-8304-a5ae53a3621b admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:05.212437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825350e0-f634-4c5b-8304-a5ae53a3621b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:05.212778 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825350e0-f634-4c5b-8304-a5ae53a3621b admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:05.213465 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825350e0-f634-4c5b-8304-a5ae53a3621b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:05.215735 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825350e0-f634-4c5b-8304-a5ae53a3621b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:05.226597 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1431/2845] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:05.231619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e295e-36e7-4890-aacf-994710cb7285 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:05.237180 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:05.243170 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e295e-36e7-4890-aacf-994710cb7285 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:05.250468 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f21d2-b888-42b0-858c-3f093a316ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:05.251202 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f21d2-b888-42b0-858c-3f093a316ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:05.251562 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f21d2-b888-42b0-858c-3f093a316ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:05.251880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f21d2-b888-42b0-858c-3f093a316ba0 admin admin] PATH_INFO: `/v3/projects/e36291540e9348dd88350d1c550dd0b9/users/30a6a750770847a2a2f9a0842bca24ef/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:05.253837 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f21d2-b888-42b0-858c-3f093a316ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e36291540e9348dd88350d1c550dd0b9, user_id=30a6a750770847a2a2f9a0842bca24ef, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:05.256497 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f21d2-b888-42b0-858c-3f093a316ba0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:05.268909 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1415/2846] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:05 2022] PUT /identity/v3/projects/e36291540e9348dd88350d1c550dd0b9/users/30a6a750770847a2a2f9a0842bca24ef/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:13:05.280368 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ff903-4475-4846-9623-14ab9ae82720 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:05.280648 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ff903-4475-4846-9623-14ab9ae82720 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:05.280913 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ff903-4475-4846-9623-14ab9ae82720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:05.310817 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-1c5ff903-4475-4846-9623-14ab9ae82720 None None] MFA Rules not processed for user `30a6a750770847a2a2f9a0842bca24ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:05.312027 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-1c5ff903-4475-4846-9623-14ab9ae82720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:05.383134 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1432/2847] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:05.395015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f21d2-b888-42b0-858c-3f093a316ba0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:05.396729 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-291f21d2-b888-42b0-858c-3f093a316ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:05.454844 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f21d2-b888-42b0-858c-3f093a316ba0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:05.466551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38e2b0-1d26-4bdd-b227-a4d1964344c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:05.467160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38e2b0-1d26-4bdd-b227-a4d1964344c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:05.467441 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38e2b0-1d26-4bdd-b227-a4d1964344c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:05.467716 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38e2b0-1d26-4bdd-b227-a4d1964344c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:05.477251 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38e2b0-1d26-4bdd-b227-a4d1964344c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:05.478658 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38e2b0-1d26-4bdd-b227-a4d1964344c0 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:05.488106 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1416/2848] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:05.569889 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ff903-4475-4846-9623-14ab9ae82720 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:05.579916 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ff903-4475-4846-9623-14ab9ae82720 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:05.583517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e496a-fecf-4915-97d9-240270d6714e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:05.584113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e496a-fecf-4915-97d9-240270d6714e service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:05.584399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e496a-fecf-4915-97d9-240270d6714e service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:05.584669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e496a-fecf-4915-97d9-240270d6714e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:05.642090 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e496a-fecf-4915-97d9-240270d6714e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:05.643453 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e496a-fecf-4915-97d9-240270d6714e service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:05.652756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1433/2849] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:10.359762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38e2b0-1d26-4bdd-b227-a4d1964344c0 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:10.422550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38e2b0-1d26-4bdd-b227-a4d1964344c0 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:10.427431 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:10.428060 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:10.428342 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:10.428617 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] PATH_INFO: `/v3/users/405b88aaaedf441ba5cff4dec8f0d1c8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:10.429753 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405b88aaaedf441ba5cff4dec8f0d1c8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:10.431651 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:10.476579 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405b88aaaedf441ba5cff4dec8f0d1c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:10.484563 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405b88aaaedf441ba5cff4dec8f0d1c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:10.485217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405b88aaaedf441ba5cff4dec8f0d1c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:10.485820 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405b88aaaedf441ba5cff4dec8f0d1c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:10.497583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405b88aaaedf441ba5cff4dec8f0d1c8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:10.509460 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1417/2850] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:10 2022] DELETE /identity/v3/users/405b88aaaedf441ba5cff4dec8f0d1c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:10.514522 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bebf3-f665-4227-9abd-d1c138d8ce9f None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:10.515037 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bebf3-f665-4227-9abd-d1c138d8ce9f None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:10.515149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bebf3-f665-4227-9abd-d1c138d8ce9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:10.534673 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-ab0bebf3-f665-4227-9abd-d1c138d8ce9f None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:10.535875 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ab0bebf3-f665-4227-9abd-d1c138d8ce9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:10.575362 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=100 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:10.595042 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1434/2851] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:10.604662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:10.606286 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:10.620226 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:10.665854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89c410-97d1-4377-a0d4-bba632b77399 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:10.670677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e173e-9bcf-438f-b51e-1a401e526362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:10.671274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e173e-9bcf-438f-b51e-1a401e526362 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:10.671552 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e173e-9bcf-438f-b51e-1a401e526362 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:10.671821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e173e-9bcf-438f-b51e-1a401e526362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:10.680908 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e173e-9bcf-438f-b51e-1a401e526362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:10.682344 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e173e-9bcf-438f-b51e-1a401e526362 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:10.691880 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1418/2852] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:10.968264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bebf3-f665-4227-9abd-d1c138d8ce9f None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:10.969784 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-ab0bebf3-f665-4227-9abd-d1c138d8ce9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:10.983473 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bebf3-f665-4227-9abd-d1c138d8ce9f None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:10.988378 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:10.989129 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:10.989477 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:10.989825 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] PATH_INFO: `/v3/projects/8a8fdea8cdf64e47b3897bacb2300986` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:10.990994 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a8fdea8cdf64e47b3897bacb2300986)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:10.993024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:11.044223 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a8fdea8cdf64e47b3897bacb2300986', 'request_id': 'req-51c4631c-8248-4cba-992a-85f9043e9219'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.050459 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a8fdea8cdf64e47b3897bacb2300986', 'request_id': 'req-51c4631c-8248-4cba-992a-85f9043e9219'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.058372 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a8fdea8cdf64e47b3897bacb2300986', 'request_id': 'req-51c4631c-8248-4cba-992a-85f9043e9219'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.068820 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] The token cache is being invalidate because project 8a8fdea8cdf64e47b3897bacb2300986 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:11.069139 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.069764 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.071336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1435/2853] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:10 2022] DELETE /identity/v3/projects/8a8fdea8cdf64e47b3897bacb2300986 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:11.076558 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e173e-9bcf-438f-b51e-1a401e526362 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:11.086069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e173e-9bcf-438f-b51e-1a401e526362 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:11.144143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:11.144768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:11.145045 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:11.145345 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] PATH_INFO: `/v3/users/30a6a750770847a2a2f9a0842bca24ef` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:11.146524 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30a6a750770847a2a2f9a0842bca24ef)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:11.148619 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:11.192771 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30a6a750770847a2a2f9a0842bca24ef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.200662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30a6a750770847a2a2f9a0842bca24ef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.201315 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30a6a750770847a2a2f9a0842bca24ef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.201919 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30a6a750770847a2a2f9a0842bca24ef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.213530 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67461010-1a92-4c87-a82e-88594bb6f8d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30a6a750770847a2a2f9a0842bca24ef'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.225911 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1419/2854] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:11 2022] DELETE /identity/v3/users/30a6a750770847a2a2f9a0842bca24ef => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:11.503808 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:11.505519 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:11.520142 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4631c-8248-4cba-992a-85f9043e9219 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:11.579869 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:11.580491 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:11.580766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:11.581032 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] PATH_INFO: `/v3/projects/e36291540e9348dd88350d1c550dd0b9` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:11.582170 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e36291540e9348dd88350d1c550dd0b9)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:11.584078 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:11.630560 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e36291540e9348dd88350d1c550dd0b9', 'request_id': 'req-377da133-8bb7-4d73-80c8-01aece376c02'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.636659 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e36291540e9348dd88350d1c550dd0b9', 'request_id': 'req-377da133-8bb7-4d73-80c8-01aece376c02'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.643890 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e36291540e9348dd88350d1c550dd0b9', 'request_id': 'req-377da133-8bb7-4d73-80c8-01aece376c02'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.650893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:11.662138 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] The token cache is being invalidate because project e36291540e9348dd88350d1c550dd0b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:11.662431 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.663082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-377da133-8bb7-4d73-80c8-01aece376c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:11.664584 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1436/2855] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:11 2022] DELETE /identity/v3/projects/e36291540e9348dd88350d1c550dd0b9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:11.670015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e3e7-5917-455e-80e9-3464f2730940 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:11.670500 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e3e7-5917-455e-80e9-3464f2730940 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:11.670605 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e3e7-5917-455e-80e9-3464f2730940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:11.690442 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1cc3e3e7-5917-455e-80e9-3464f2730940 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:11.691635 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1cc3e3e7-5917-455e-80e9-3464f2730940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:11.719616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:11.752188 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1420/2856] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:11.759916 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf7a5c-f9ef-4418-bd48-bb988c5c9b9d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:11.760199 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf7a5c-f9ef-4418-bd48-bb988c5c9b9d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:11.760466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf7a5c-f9ef-4418-bd48-bb988c5c9b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:11.780420 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-21cf7a5c-f9ef-4418-bd48-bb988c5c9b9d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:11.781563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-21cf7a5c-f9ef-4418-bd48-bb988c5c9b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:11.789465 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1437/2857] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:11.794253 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3e3e7-5917-455e-80e9-3464f2730940 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:11.803708 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3e3e7-5917-455e-80e9-3464f2730940 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:11.808182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839eea06-15e8-47b3-9051-a6b71d5ef2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:11.809469 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839eea06-15e8-47b3-9051-a6b71d5ef2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:11.809841 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839eea06-15e8-47b3-9051-a6b71d5ef2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:11.810218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839eea06-15e8-47b3-9051-a6b71d5ef2f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:11.810856 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839eea06-15e8-47b3-9051-a6b71d5ef2f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:13:11.811156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839eea06-15e8-47b3-9051-a6b71d5ef2f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:11.812804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839eea06-15e8-47b3-9051-a6b71d5ef2f3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:11.825417 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1421/2858] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:13:11.829730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf7a5c-f9ef-4418-bd48-bb988c5c9b9d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:11.838930 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf7a5c-f9ef-4418-bd48-bb988c5c9b9d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:11.843668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8b25-37d9-4be6-be7b-f7896937ffe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:11.844250 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8b25-37d9-4be6-be7b-f7896937ffe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:11.844527 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8b25-37d9-4be6-be7b-f7896937ffe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:11.844799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8b25-37d9-4be6-be7b-f7896937ffe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:11.845467 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d8b25-37d9-4be6-be7b-f7896937ffe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:11.847395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d8b25-37d9-4be6-be7b-f7896937ffe7 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:11.876096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1438/2859] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:11 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:13:11.880378 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839eea06-15e8-47b3-9051-a6b71d5ef2f3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:11.890814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839eea06-15e8-47b3-9051-a6b71d5ef2f3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:11.946921 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba8870-3a68-4c5e-b8a8-9f5dbe8f456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:11.947511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba8870-3a68-4c5e-b8a8-9f5dbe8f456c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:11.947790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba8870-3a68-4c5e-b8a8-9f5dbe8f456c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:11.948062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba8870-3a68-4c5e-b8a8-9f5dbe8f456c admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:11.948692 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba8870-3a68-4c5e-b8a8-9f5dbe8f456c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:11.950981 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba8870-3a68-4c5e-b8a8-9f5dbe8f456c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:11.972811 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1422/2860] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:11 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:11.976919 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8b25-37d9-4be6-be7b-f7896937ffe7 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:11.988171 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8b25-37d9-4be6-be7b-f7896937ffe7 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:11.992711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d49b7-4055-4bbc-9d36-082d6c4da863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:11.993326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d49b7-4055-4bbc-9d36-082d6c4da863 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:11.993600 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d49b7-4055-4bbc-9d36-082d6c4da863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:11.993862 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d49b7-4055-4bbc-9d36-082d6c4da863 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:11.994490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d49b7-4055-4bbc-9d36-082d6c4da863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:11.996073 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d49b7-4055-4bbc-9d36-082d6c4da863 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.005753 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1439/2861] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:12.009717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba8870-3a68-4c5e-b8a8-9f5dbe8f456c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.020116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba8870-3a68-4c5e-b8a8-9f5dbe8f456c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.024999 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b812-133a-4802-9f4d-74703ef5027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.025624 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b812-133a-4802-9f4d-74703ef5027a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.025901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b812-133a-4802-9f4d-74703ef5027a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.026169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b812-133a-4802-9f4d-74703ef5027a admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.026804 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b812-133a-4802-9f4d-74703ef5027a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.028985 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b812-133a-4802-9f4d-74703ef5027a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.040685 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1423/2862] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:12.045345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d49b7-4055-4bbc-9d36-082d6c4da863 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.055097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d49b7-4055-4bbc-9d36-082d6c4da863 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.059971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df8daa-a146-44ff-bddb-4b6d56c3ed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.060618 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df8daa-a146-44ff-bddb-4b6d56c3ed6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.060897 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df8daa-a146-44ff-bddb-4b6d56c3ed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.061191 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df8daa-a146-44ff-bddb-4b6d56c3ed6d admin admin] PATH_INFO: `/v3/projects/819a6f98b9b04637b50ef3eff4b4d17d/users/d628689a8c584f299492d6837e792337/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.077020 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df8daa-a146-44ff-bddb-4b6d56c3ed6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=819a6f98b9b04637b50ef3eff4b4d17d, user_id=d628689a8c584f299492d6837e792337, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.079699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df8daa-a146-44ff-bddb-4b6d56c3ed6d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.094552 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1440/2863] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:12 2022] PUT /identity/v3/projects/819a6f98b9b04637b50ef3eff4b4d17d/users/d628689a8c584f299492d6837e792337/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:12.104831 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dad03-f784-441f-8084-9bc30d956920 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.105147 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dad03-f784-441f-8084-9bc30d956920 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.105426 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dad03-f784-441f-8084-9bc30d956920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.133187 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-fc0dad03-f784-441f-8084-9bc30d956920 None None] MFA Rules not processed for user `d628689a8c584f299492d6837e792337`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:12.134399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fc0dad03-f784-441f-8084-9bc30d956920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:12.198807 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1424/2864] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:12.203972 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df8daa-a146-44ff-bddb-4b6d56c3ed6d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.264221 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df8daa-a146-44ff-bddb-4b6d56c3ed6d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.268810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14f4e9-bbe1-45d9-831e-283c85acbb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.269436 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14f4e9-bbe1-45d9-831e-283c85acbb89 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.269710 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14f4e9-bbe1-45d9-831e-283c85acbb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.269977 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14f4e9-bbe1-45d9-831e-283c85acbb89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.270609 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14f4e9-bbe1-45d9-831e-283c85acbb89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.272757 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14f4e9-bbe1-45d9-831e-283c85acbb89 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.300823 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1441/2865] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:12 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:12.304988 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0dad03-f784-441f-8084-9bc30d956920 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.316228 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0dad03-f784-441f-8084-9bc30d956920 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.341609 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:12.373393 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e487f-b81c-47ec-974f-46f594463f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.373993 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e487f-b81c-47ec-974f-46f594463f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.374277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e487f-b81c-47ec-974f-46f594463f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.374550 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e487f-b81c-47ec-974f-46f594463f03 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.375167 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e487f-b81c-47ec-974f-46f594463f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.377064 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e487f-b81c-47ec-974f-46f594463f03 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.399032 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1425/2866] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:12 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:12.403534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14f4e9-bbe1-45d9-831e-283c85acbb89 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.413903 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14f4e9-bbe1-45d9-831e-283c85acbb89 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.419461 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9794243-8c32-4eac-894e-fe79aed4f37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.420052 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794243-8c32-4eac-894e-fe79aed4f37d admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.420333 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794243-8c32-4eac-894e-fe79aed4f37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.420599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794243-8c32-4eac-894e-fe79aed4f37d admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.421260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9794243-8c32-4eac-894e-fe79aed4f37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.422848 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9794243-8c32-4eac-894e-fe79aed4f37d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.433573 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1442/2867] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:12.438410 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e487f-b81c-47ec-974f-46f594463f03 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.448154 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e487f-b81c-47ec-974f-46f594463f03 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.452662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558666f9-6054-4699-9b19-a1f6f4ed03bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.453322 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558666f9-6054-4699-9b19-a1f6f4ed03bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.453595 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558666f9-6054-4699-9b19-a1f6f4ed03bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.453865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558666f9-6054-4699-9b19-a1f6f4ed03bc admin admin] PATH_INFO: `/v3/projects/f1ae2a724f1844c3b2188265c75048a3/users/dc9614dd1b754d5b8ecdadadea3dd4d9/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.468043 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558666f9-6054-4699-9b19-a1f6f4ed03bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ae2a724f1844c3b2188265c75048a3, user_id=dc9614dd1b754d5b8ecdadadea3dd4d9, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.470725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558666f9-6054-4699-9b19-a1f6f4ed03bc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.483790 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1426/2868] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:12 2022] PUT /identity/v3/projects/f1ae2a724f1844c3b2188265c75048a3/users/dc9614dd1b754d5b8ecdadadea3dd4d9/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:12.487873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9794243-8c32-4eac-894e-fe79aed4f37d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.498474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9794243-8c32-4eac-894e-fe79aed4f37d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.556949 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96cfae-40e6-448b-b0bc-ad48b8441e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.557598 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96cfae-40e6-448b-b0bc-ad48b8441e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.557879 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96cfae-40e6-448b-b0bc-ad48b8441e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.558144 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96cfae-40e6-448b-b0bc-ad48b8441e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.559699 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96cfae-40e6-448b-b0bc-ad48b8441e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.561326 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96cfae-40e6-448b-b0bc-ad48b8441e67 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.571407 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1443/2869] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:12.575865 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558666f9-6054-4699-9b19-a1f6f4ed03bc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.586316 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558666f9-6054-4699-9b19-a1f6f4ed03bc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.591559 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42455d7-7d11-4b7b-b7bc-30cc543390ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.592208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42455d7-7d11-4b7b-b7bc-30cc543390ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.592489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42455d7-7d11-4b7b-b7bc-30cc543390ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.592769 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42455d7-7d11-4b7b-b7bc-30cc543390ae admin admin] PATH_INFO: `/v3/projects/f1ae2a724f1844c3b2188265c75048a3/users/dc9614dd1b754d5b8ecdadadea3dd4d9/roles/7e63743509de412fbbf1c7b60a8ae2ce` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.594755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42455d7-7d11-4b7b-b7bc-30cc543390ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ae2a724f1844c3b2188265c75048a3, user_id=dc9614dd1b754d5b8ecdadadea3dd4d9, role_id=7e63743509de412fbbf1c7b60a8ae2ce)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.597438 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42455d7-7d11-4b7b-b7bc-30cc543390ae admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.608272 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1427/2870] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:12 2022] PUT /identity/v3/projects/f1ae2a724f1844c3b2188265c75048a3/users/dc9614dd1b754d5b8ecdadadea3dd4d9/roles/7e63743509de412fbbf1c7b60a8ae2ce => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:13:12.619403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf7c31-110a-46e7-8df4-8a54b1acabdc None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.619690 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf7c31-110a-46e7-8df4-8a54b1acabdc None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.619964 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf7c31-110a-46e7-8df4-8a54b1acabdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.628594 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:12.650101 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-28cf7c31-110a-46e7-8df4-8a54b1acabdc None None] MFA Rules not processed for user `dc9614dd1b754d5b8ecdadadea3dd4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:12.651305 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-28cf7c31-110a-46e7-8df4-8a54b1acabdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:12.717785 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1444/2871] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:12.729693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42455d7-7d11-4b7b-b7bc-30cc543390ae admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.732346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a42455d7-7d11-4b7b-b7bc-30cc543390ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:12.789762 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42455d7-7d11-4b7b-b7bc-30cc543390ae admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.794489 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c172a3f-5058-43f4-a936-364a0668ca1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.795094 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c172a3f-5058-43f4-a936-364a0668ca1c service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.795372 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c172a3f-5058-43f4-a936-364a0668ca1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.795639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c172a3f-5058-43f4-a936-364a0668ca1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.804533 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c172a3f-5058-43f4-a936-364a0668ca1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.806029 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c172a3f-5058-43f4-a936-364a0668ca1c service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.815821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1428/2872] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:12.890543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf7c31-110a-46e7-8df4-8a54b1acabdc None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:12.900535 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf7c31-110a-46e7-8df4-8a54b1acabdc None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:12.904617 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586316d-9c90-4b18-aed0-5dfb5843fe7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:12.905246 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586316d-9c90-4b18-aed0-5dfb5843fe7e service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:12.905524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586316d-9c90-4b18-aed0-5dfb5843fe7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:12.905794 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586316d-9c90-4b18-aed0-5dfb5843fe7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:12.961675 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586316d-9c90-4b18-aed0-5dfb5843fe7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:12.963104 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586316d-9c90-4b18-aed0-5dfb5843fe7e service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:12.972531 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1445/2873] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:13.234048 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c172a3f-5058-43f4-a936-364a0668ca1c service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:13.296725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c172a3f-5058-43f4-a936-364a0668ca1c service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:13.301274 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:13.301895 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:13.302174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:13.302444 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] PATH_INFO: `/v3/users/d628689a8c584f299492d6837e792337` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:13.303560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d628689a8c584f299492d6837e792337)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:13.305532 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:13.334156 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:13.355392 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd628689a8c584f299492d6837e792337'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:13.368119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd628689a8c584f299492d6837e792337'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:13.368730 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd628689a8c584f299492d6837e792337'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:13.370120 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd628689a8c584f299492d6837e792337'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:13.382768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd628689a8c584f299492d6837e792337'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:13.395917 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1429/2874] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:13 2022] DELETE /identity/v3/users/d628689a8c584f299492d6837e792337 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:13.404293 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6247419-9330-4c71-9be6-70aab78d5775 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:13.404599 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6247419-9330-4c71-9be6-70aab78d5775 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:13.404870 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6247419-9330-4c71-9be6-70aab78d5775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:13.424137 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e6247419-9330-4c71-9be6-70aab78d5775 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:13.425352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e6247419-9330-4c71-9be6-70aab78d5775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:13.483439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1446/2875] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:13.492955 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:13.494403 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:13.562081 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778b448-7641-4b9f-a073-c623e77d71c0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:13.566101 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c0da9-5418-4423-bbe6-db24dd8013d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:13.567114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c0da9-5418-4423-bbe6-db24dd8013d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:13.567591 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c0da9-5418-4423-bbe6-db24dd8013d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:13.567871 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c0da9-5418-4423-bbe6-db24dd8013d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:13.578612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c0da9-5418-4423-bbe6-db24dd8013d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:13.580037 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c0da9-5418-4423-bbe6-db24dd8013d2 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:13.589742 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1430/2876] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:13.863595 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6247419-9330-4c71-9be6-70aab78d5775 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:13.865166 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e6247419-9330-4c71-9be6-70aab78d5775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:13.894315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6247419-9330-4c71-9be6-70aab78d5775 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:13.899642 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:13.900284 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:13.900563 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:13.900854 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] PATH_INFO: `/v3/projects/819a6f98b9b04637b50ef3eff4b4d17d` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:13.902019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819a6f98b9b04637b50ef3eff4b4d17d)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:13.904125 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:13.945028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:13.969608 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819a6f98b9b04637b50ef3eff4b4d17d', 'request_id': 'req-019ea582-fc26-4c70-9314-5a3398c9dac0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:13.976799 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819a6f98b9b04637b50ef3eff4b4d17d', 'request_id': 'req-019ea582-fc26-4c70-9314-5a3398c9dac0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:13.987446 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819a6f98b9b04637b50ef3eff4b4d17d', 'request_id': 'req-019ea582-fc26-4c70-9314-5a3398c9dac0'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.001380 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] The token cache is being invalidate because project 819a6f98b9b04637b50ef3eff4b4d17d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:14.001669 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.002380 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.003931 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1447/2877] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:13 2022] DELETE /identity/v3/projects/819a6f98b9b04637b50ef3eff4b4d17d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:14.008259 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c0da9-5418-4423-bbe6-db24dd8013d2 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:14.018656 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c0da9-5418-4423-bbe6-db24dd8013d2 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:14.081254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:14.081893 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:14.082169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:14.082436 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] PATH_INFO: `/v3/users/dc9614dd1b754d5b8ecdadadea3dd4d9` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:14.083578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc9614dd1b754d5b8ecdadadea3dd4d9)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:14.086189 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:14.155788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc9614dd1b754d5b8ecdadadea3dd4d9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.157637 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:14.166199 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc9614dd1b754d5b8ecdadadea3dd4d9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.167295 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc9614dd1b754d5b8ecdadadea3dd4d9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.168239 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc9614dd1b754d5b8ecdadadea3dd4d9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.186432 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3165713f-3bba-4544-bb09-b0866cd7b25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc9614dd1b754d5b8ecdadadea3dd4d9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.202817 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1431/2878] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:14 2022] DELETE /identity/v3/users/dc9614dd1b754d5b8ecdadadea3dd4d9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:14.484361 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:14.485893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:14.499596 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ea582-fc26-4c70-9314-5a3398c9dac0 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:14.557177 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:14.557804 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:14.558084 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:14.558352 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] PATH_INFO: `/v3/projects/f1ae2a724f1844c3b2188265c75048a3` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:14.560257 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ae2a724f1844c3b2188265c75048a3)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:14.562197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:14.611534 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ae2a724f1844c3b2188265c75048a3', 'request_id': 'req-274765f5-3c80-44a2-a1ab-ec9279b12b8a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.617668 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ae2a724f1844c3b2188265c75048a3', 'request_id': 'req-274765f5-3c80-44a2-a1ab-ec9279b12b8a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.625267 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ae2a724f1844c3b2188265c75048a3', 'request_id': 'req-274765f5-3c80-44a2-a1ab-ec9279b12b8a'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.635415 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] The token cache is being invalidate because project f1ae2a724f1844c3b2188265c75048a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:14.635696 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.636307 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-274765f5-3c80-44a2-a1ab-ec9279b12b8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:14.638097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1448/2879] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:14 2022] DELETE /identity/v3/projects/f1ae2a724f1844c3b2188265c75048a3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:14.643172 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3686-c6fc-4266-9cab-abdcf8828240 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:14.643462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3686-c6fc-4266-9cab-abdcf8828240 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:14.643781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3686-c6fc-4266-9cab-abdcf8828240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:14.664899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-345d3686-c6fc-4266-9cab-abdcf8828240 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:14.666136 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-345d3686-c6fc-4266-9cab-abdcf8828240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:14.724821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1432/2880] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:14.733451 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8445-539d-4921-a71d-6d36bba2696d None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:14.733732 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8445-539d-4921-a71d-6d36bba2696d None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:14.734041 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8445-539d-4921-a71d-6d36bba2696d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:14.753793 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-780d8445-539d-4921-a71d-6d36bba2696d None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:14.754980 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-780d8445-539d-4921-a71d-6d36bba2696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:14.763439 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1449/2881] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:14.767642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d3686-c6fc-4266-9cab-abdcf8828240 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:14.777848 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d3686-c6fc-4266-9cab-abdcf8828240 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:14.782500 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0efb3-aacd-4738-9b8b-f2ffebce2e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:14.783093 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0efb3-aacd-4738-9b8b-f2ffebce2e95 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:14.783371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0efb3-aacd-4738-9b8b-f2ffebce2e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:14.783642 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0efb3-aacd-4738-9b8b-f2ffebce2e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:14.784281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0efb3-aacd-4738-9b8b-f2ffebce2e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:13:14.784584 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0efb3-aacd-4738-9b8b-f2ffebce2e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:14.786208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0efb3-aacd-4738-9b8b-f2ffebce2e95 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:14.798284 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1433/2882] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:13:14.802525 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8445-539d-4921-a71d-6d36bba2696d None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:14.812341 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8445-539d-4921-a71d-6d36bba2696d None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:14.816907 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfb9ad-b514-4abb-a5d5-585b2b7c0490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:14.817550 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfb9ad-b514-4abb-a5d5-585b2b7c0490 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:14.817846 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfb9ad-b514-4abb-a5d5-585b2b7c0490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:14.818071 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfb9ad-b514-4abb-a5d5-585b2b7c0490 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:14.818693 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfb9ad-b514-4abb-a5d5-585b2b7c0490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:14.820549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfb9ad-b514-4abb-a5d5-585b2b7c0490 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:14.846403 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1450/2883] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:14 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:14.850544 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0efb3-aacd-4738-9b8b-f2ffebce2e95 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:14.861789 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0efb3-aacd-4738-9b8b-f2ffebce2e95 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:14.917346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3dcc5-a755-4fb4-988b-c7a0bd00a946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:14.917946 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3dcc5-a755-4fb4-988b-c7a0bd00a946 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:14.918791 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3dcc5-a755-4fb4-988b-c7a0bd00a946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:14.919114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3dcc5-a755-4fb4-988b-c7a0bd00a946 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:14.919755 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3dcc5-a755-4fb4-988b-c7a0bd00a946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:14.921678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3dcc5-a755-4fb4-988b-c7a0bd00a946 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:14.944476 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1434/2884] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:14 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:14.948611 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfb9ad-b514-4abb-a5d5-585b2b7c0490 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:14.959285 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfb9ad-b514-4abb-a5d5-585b2b7c0490 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:14.963850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a3310-f08f-4892-8d24-09f0d1ed6dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:14.964444 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a3310-f08f-4892-8d24-09f0d1ed6dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:14.964716 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a3310-f08f-4892-8d24-09f0d1ed6dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:14.965711 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a3310-f08f-4892-8d24-09f0d1ed6dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:14.966664 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a3310-f08f-4892-8d24-09f0d1ed6dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:14.968253 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a3310-f08f-4892-8d24-09f0d1ed6dd4 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:14.978483 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1451/2885] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:14.983522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3dcc5-a755-4fb4-988b-c7a0bd00a946 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:14.994234 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3dcc5-a755-4fb4-988b-c7a0bd00a946 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:14.999083 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2c594-cacf-4333-af18-cd7a39d2fdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:14.999677 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c594-cacf-4333-af18-cd7a39d2fdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:15.000099 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c594-cacf-4333-af18-cd7a39d2fdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:15.000371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c594-cacf-4333-af18-cd7a39d2fdd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:15.001015 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2c594-cacf-4333-af18-cd7a39d2fdd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:15.002686 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2c594-cacf-4333-af18-cd7a39d2fdd5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:15.020124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1435/2886] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:15.024113 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a3310-f08f-4892-8d24-09f0d1ed6dd4 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:15.029991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:15.036331 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a3310-f08f-4892-8d24-09f0d1ed6dd4 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:15.042028 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b226d02-688b-4ee3-9e2c-6ad58e3a4b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:15.042723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b226d02-688b-4ee3-9e2c-6ad58e3a4b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:15.043000 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b226d02-688b-4ee3-9e2c-6ad58e3a4b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:15.043835 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b226d02-688b-4ee3-9e2c-6ad58e3a4b8d admin admin] PATH_INFO: `/v3/projects/0fc156b7a56e4ecc999a85d59bd008b3/users/1df3f20605ad4ccfaefddb64cec11875/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:15.058506 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b226d02-688b-4ee3-9e2c-6ad58e3a4b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc156b7a56e4ecc999a85d59bd008b3, user_id=1df3f20605ad4ccfaefddb64cec11875, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:15.061565 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b226d02-688b-4ee3-9e2c-6ad58e3a4b8d admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:15.074259 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1452/2887] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:15 2022] PUT /identity/v3/projects/0fc156b7a56e4ecc999a85d59bd008b3/users/1df3f20605ad4ccfaefddb64cec11875/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:15.079523 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af948-763f-42b1-a7ce-ecbf198bf6bd None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:15.079806 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af948-763f-42b1-a7ce-ecbf198bf6bd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:15.080074 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af948-763f-42b1-a7ce-ecbf198bf6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:15.099570 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-238af948-763f-42b1-a7ce-ecbf198bf6bd None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:15.100766 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-238af948-763f-42b1-a7ce-ecbf198bf6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:15.127182 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:15.163232 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1436/2888] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:15.173509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b226d02-688b-4ee3-9e2c-6ad58e3a4b8d admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:15.175140 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-6b226d02-688b-4ee3-9e2c-6ad58e3a4b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:15.231435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b226d02-688b-4ee3-9e2c-6ad58e3a4b8d admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:15.235300 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2e739-2c01-4282-af44-1521aa396f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:15.235900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2e739-2c01-4282-af44-1521aa396f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:15.236173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2e739-2c01-4282-af44-1521aa396f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:15.236440 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2e739-2c01-4282-af44-1521aa396f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:15.245615 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2e739-2c01-4282-af44-1521aa396f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:15.247513 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2e739-2c01-4282-af44-1521aa396f0d service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:15.256920 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1453/2889] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:15.874590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76c39c-0591-454f-941c-2cf36e20ed53 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:15.874950 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76c39c-0591-454f-941c-2cf36e20ed53 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:15.875225 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76c39c-0591-454f-941c-2cf36e20ed53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:15.903582 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ee76c39c-0591-454f-941c-2cf36e20ed53 None None] MFA Rules not processed for user `1df3f20605ad4ccfaefddb64cec11875`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:15.904801 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ee76c39c-0591-454f-941c-2cf36e20ed53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:15.972764 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1437/2890] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:15.986153 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2e739-2c01-4282-af44-1521aa396f0d service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:15.987553 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-59b2e739-2c01-4282-af44-1521aa396f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:16.047489 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2e739-2c01-4282-af44-1521aa396f0d service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:16.051499 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091727cd-def9-4e65-bc71-fa0c160bf845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:16.052260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091727cd-def9-4e65-bc71-fa0c160bf845 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:16.052260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091727cd-def9-4e65-bc71-fa0c160bf845 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:16.052260 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091727cd-def9-4e65-bc71-fa0c160bf845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:16.061390 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091727cd-def9-4e65-bc71-fa0c160bf845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:16.062844 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091727cd-def9-4e65-bc71-fa0c160bf845 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:16.071971 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1454/2891] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:13:17.430161 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f4b6-25ec-4e26-8e88-c3e35d0baaf4 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:17.430705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f4b6-25ec-4e26-8e88-c3e35d0baaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:17.430705 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f4b6-25ec-4e26-8e88-c3e35d0baaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:17.450636 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a5a3f4b6-25ec-4e26-8e88-c3e35d0baaf4 None None] MFA Rules not processed for user `1df3f20605ad4ccfaefddb64cec11875`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:17.451821 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a5a3f4b6-25ec-4e26-8e88-c3e35d0baaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:17.458578 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1438/2892] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:13:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:17.697377 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091727cd-def9-4e65-bc71-fa0c160bf845 service cinder] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:17.698873 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-091727cd-def9-4e65-bc71-fa0c160bf845 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:17.710911 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091727cd-def9-4e65-bc71-fa0c160bf845 service cinder] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:17.714438 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:17.715086 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:17.715425 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:17.715730 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:17.717399 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:17.726809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:17.728264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:17.737280 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1455/2893] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:27.445124 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=48 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:27.733400 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:45.622317 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3f4b6-25ec-4e26-8e88-c3e35d0baaf4 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:45.623818 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a5a3f4b6-25ec-4e26-8e88-c3e35d0baaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:45.710376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3f4b6-25ec-4e26-8e88-c3e35d0baaf4 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:45.716218 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9b88c495fcb47a289a6a45276c891e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:45.716816 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:45.717121 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:45.717402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:45.719166 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:45.731727 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:45.733416 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:45.744196 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1439/2894] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:49.813934 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:49.818482 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:49.909758 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2f7ba-138d-44ac-b968-480b9c34c0b4 service neutron] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:49.915024 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c0673-2808-44e8-ac27-b30f6d4d1b68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:49.915647 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c0673-2808-44e8-ac27-b30f6d4d1b68 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:49.915926 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c0673-2808-44e8-ac27-b30f6d4d1b68 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:49.916197 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c0673-2808-44e8-ac27-b30f6d4d1b68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:49.918045 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-b22c0673-2808-44e8-ac27-b30f6d4d1b68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:49.931558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c0673-2808-44e8-ac27-b30f6d4d1b68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:49.933211 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c0673-2808-44e8-ac27-b30f6d4d1b68 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:49.946756 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1456/2895] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:53.406240 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:53.407782 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:53.423230 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97e64-d39f-428b-a8ab-f549ffc20c4f service nova] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.427839 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.428458 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.428738 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.429010 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] PATH_INFO: `/v3/users/21a84ede2a594f7ab23b442baa9f2f5a` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.430158 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a84ede2a594f7ab23b442baa9f2f5a)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.432047 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:53.477380 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a84ede2a594f7ab23b442baa9f2f5a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.487162 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a84ede2a594f7ab23b442baa9f2f5a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.487924 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a84ede2a594f7ab23b442baa9f2f5a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.488526 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a84ede2a594f7ab23b442baa9f2f5a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.500590 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a84ede2a594f7ab23b442baa9f2f5a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.512943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1440/2896] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:53 2022] DELETE /identity/v3/users/21a84ede2a594f7ab23b442baa9f2f5a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:53.684189 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c0673-2808-44e8-ac27-b30f6d4d1b68 service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:53.697047 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c0673-2808-44e8-ac27-b30f6d4d1b68 service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.756866 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.757530 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.757809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.758082 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] PATH_INFO: `/v3/projects/aa4d639296c34f1dbf0b327fd0828620` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.759194 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4d639296c34f1dbf0b327fd0828620)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.761124 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:53.809863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4d639296c34f1dbf0b327fd0828620', 'request_id': 'req-509db6ab-b7e3-49c5-8f28-229a34338464'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.815834 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4d639296c34f1dbf0b327fd0828620', 'request_id': 'req-509db6ab-b7e3-49c5-8f28-229a34338464'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.824217 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4d639296c34f1dbf0b327fd0828620', 'request_id': 'req-509db6ab-b7e3-49c5-8f28-229a34338464'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.836520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] The token cache is being invalidate because project aa4d639296c34f1dbf0b327fd0828620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:53.836801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.837445 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.838948 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1457/2897] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:53 2022] DELETE /identity/v3/projects/aa4d639296c34f1dbf0b327fd0828620 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:53.843600 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:53.845217 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:53.859145 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198a113-7ff3-4d4f-99bd-cdee3336c080 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.910376 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:53.917472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.918293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.918966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.919354 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] PATH_INFO: `/v3/users/f65ef4736d384d0ea7ef783a789296b0` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.920693 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f65ef4736d384d0ea7ef783a789296b0)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.922979 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:53.976174 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f65ef4736d384d0ea7ef783a789296b0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.984056 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f65ef4736d384d0ea7ef783a789296b0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.984725 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f65ef4736d384d0ea7ef783a789296b0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.985364 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f65ef4736d384d0ea7ef783a789296b0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.998271 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-dbe54134-1623-4832-86c8-f7dfd721cd88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f65ef4736d384d0ea7ef783a789296b0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:54.010078 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1441/2898] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:53 2022] DELETE /identity/v3/users/f65ef4736d384d0ea7ef783a789296b0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:54.189411 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:54.190880 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:54.204054 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509db6ab-b7e3-49c5-8f28-229a34338464 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:54.222353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:54.270097 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:54.270731 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:54.271009 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:54.271278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] PATH_INFO: `/v3/projects/0f2f7592e46b42e2bfcaadde11a53d56` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:54.272395 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f2f7592e46b42e2bfcaadde11a53d56)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:54.274637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:54.321893 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f2f7592e46b42e2bfcaadde11a53d56', 'request_id': 'req-c451a571-90b7-499c-84d8-541266415462'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:54.328112 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f2f7592e46b42e2bfcaadde11a53d56', 'request_id': 'req-c451a571-90b7-499c-84d8-541266415462'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:54.334679 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f2f7592e46b42e2bfcaadde11a53d56', 'request_id': 'req-c451a571-90b7-499c-84d8-541266415462'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:54.345780 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] The token cache is being invalidate because project 0f2f7592e46b42e2bfcaadde11a53d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:54.346065 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:54.346687 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:54.348156 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1458/2899] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:54 2022] DELETE /identity/v3/projects/0f2f7592e46b42e2bfcaadde11a53d56 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:04.006717 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:04.342770 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:19.763079 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931873c-52f2-4125-bc5d-f6ece22ba2d9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:19.763633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931873c-52f2-4125-bc5d-f6ece22ba2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:19.763633 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931873c-52f2-4125-bc5d-f6ece22ba2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:19.783399 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8931873c-52f2-4125-bc5d-f6ece22ba2d9 None None] MFA Rules not processed for user `1df3f20605ad4ccfaefddb64cec11875`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:19.784588 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8931873c-52f2-4125-bc5d-f6ece22ba2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:19.839899 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1442/2900] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:14:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:24.720850 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:24.722401 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:24.799822 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451a571-90b7-499c-84d8-541266415462 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:24.805946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59e6dc-ae3f-4743-93a8-704fe8183c40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:24.807529 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e6dc-ae3f-4743-93a8-704fe8183c40 service placement] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:24.808034 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e6dc-ae3f-4743-93a8-704fe8183c40 service placement] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:24.808589 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e6dc-ae3f-4743-93a8-704fe8183c40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:24.821490 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5a59e6dc-ae3f-4743-93a8-704fe8183c40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:24.824455 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59e6dc-ae3f-4743-93a8-704fe8183c40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:24.826849 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59e6dc-ae3f-4743-93a8-704fe8183c40 service placement] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:24.837462 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1459/2901] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:29.833277 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=25 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:34.833017 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=32 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:21.502854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931873c-52f2-4125-bc5d-f6ece22ba2d9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:21.504331 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8931873c-52f2-4125-bc5d-f6ece22ba2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:21.570411 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931873c-52f2-4125-bc5d-f6ece22ba2d9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:21.575062 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:21.575814 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:21.576089 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:21.576375 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] PATH_INFO: `/v3/users/1df3f20605ad4ccfaefddb64cec11875` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:21.577511 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df3f20605ad4ccfaefddb64cec11875)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:21.579419 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:21.626487 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df3f20605ad4ccfaefddb64cec11875'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.634169 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df3f20605ad4ccfaefddb64cec11875'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.634875 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df3f20605ad4ccfaefddb64cec11875'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.635521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df3f20605ad4ccfaefddb64cec11875'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.647947 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-56387ba3-2731-4fbb-b59c-6f778c075121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df3f20605ad4ccfaefddb64cec11875'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.661207 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1443/2902] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:15:21 2022] DELETE /identity/v3/users/1df3f20605ad4ccfaefddb64cec11875 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:15:21.833247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59e6dc-ae3f-4743-93a8-704fe8183c40 service placement] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:21.847247 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59e6dc-ae3f-4743-93a8-704fe8183c40 service placement] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:21.904900 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:21.905543 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:21.905814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:21.906077 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] PATH_INFO: `/v3/projects/0fc156b7a56e4ecc999a85d59bd008b3` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:21.907173 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fc156b7a56e4ecc999a85d59bd008b3)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:21.909105 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:21.954232 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fc156b7a56e4ecc999a85d59bd008b3', 'request_id': 'req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.960149 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fc156b7a56e4ecc999a85d59bd008b3', 'request_id': 'req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.968322 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fc156b7a56e4ecc999a85d59bd008b3', 'request_id': 'req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.979549 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] The token cache is being invalidate because project 0fc156b7a56e4ecc999a85d59bd008b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:15:21.979837 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.980466 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-f1b3c84c-48ac-43e0-9c47-cfe05d77966e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:21.982743 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1460/2903] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:15:21 2022] DELETE /identity/v3/projects/0fc156b7a56e4ecc999a85d59bd008b3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:21.987616 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac880d-0024-4ee9-87e4-a4ef41e6a3a5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:21.987901 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac880d-0024-4ee9-87e4-a4ef41e6a3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:21.988173 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac880d-0024-4ee9-87e4-a4ef41e6a3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.007330 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-57ac880d-0024-4ee9-87e4-a4ef41e6a3a5 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:22.008522 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-57ac880d-0024-4ee9-87e4-a4ef41e6a3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:22.067542 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1444/2904] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:22.075723 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7a2f0-79fc-4aa2-86de-4d5993047013 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:22.076006 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7a2f0-79fc-4aa2-86de-4d5993047013 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:22.076275 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7a2f0-79fc-4aa2-86de-4d5993047013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.096044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-e9c7a2f0-79fc-4aa2-86de-4d5993047013 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:22.097271 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-e9c7a2f0-79fc-4aa2-86de-4d5993047013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:22.104974 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1461/2905] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:22.109122 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac880d-0024-4ee9-87e4-a4ef41e6a3a5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:22.119066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac880d-0024-4ee9-87e4-a4ef41e6a3a5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:22.123689 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83450630-872e-47b9-adb5-e7b0720693f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:22.124275 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83450630-872e-47b9-adb5-e7b0720693f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:22.124548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83450630-872e-47b9-adb5-e7b0720693f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:22.124813 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83450630-872e-47b9-adb5-e7b0720693f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.125477 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83450630-872e-47b9-adb5-e7b0720693f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:15:22.125774 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83450630-872e-47b9-adb5-e7b0720693f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:22.127335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83450630-872e-47b9-adb5-e7b0720693f0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:22.139371 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1445/2906] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:15:22.143781 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7a2f0-79fc-4aa2-86de-4d5993047013 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:22.154437 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7a2f0-79fc-4aa2-86de-4d5993047013 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:22.159079 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8208-4390-4fa5-a0f3-7b2d8360912a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:22.159665 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8208-4390-4fa5-a0f3-7b2d8360912a admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:22.159942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8208-4390-4fa5-a0f3-7b2d8360912a admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:22.160315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8208-4390-4fa5-a0f3-7b2d8360912a admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.160942 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f8208-4390-4fa5-a0f3-7b2d8360912a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:22.162857 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f8208-4390-4fa5-a0f3-7b2d8360912a admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:22.190660 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1462/2907] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:15:22 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:15:22.194597 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83450630-872e-47b9-adb5-e7b0720693f0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:22.205276 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83450630-872e-47b9-adb5-e7b0720693f0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:22.208970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:22.265517 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047eb93-5f95-472b-a977-ad328febc6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:22.266119 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047eb93-5f95-472b-a977-ad328febc6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:22.266394 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047eb93-5f95-472b-a977-ad328febc6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:22.266662 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047eb93-5f95-472b-a977-ad328febc6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.267298 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047eb93-5f95-472b-a977-ad328febc6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:22.269203 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047eb93-5f95-472b-a977-ad328febc6c2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:22.290356 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1446/2908] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:15:22 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:15:22.295161 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8208-4390-4fa5-a0f3-7b2d8360912a admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:22.305330 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8208-4390-4fa5-a0f3-7b2d8360912a admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:22.310176 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c97e43-11b2-4fd7-9daa-164f421686c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:22.310762 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c97e43-11b2-4fd7-9daa-164f421686c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:22.311038 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c97e43-11b2-4fd7-9daa-164f421686c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:22.311304 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c97e43-11b2-4fd7-9daa-164f421686c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.311935 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c97e43-11b2-4fd7-9daa-164f421686c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:22.313571 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c97e43-11b2-4fd7-9daa-164f421686c9 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:22.323345 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1463/2909] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:15:22.327788 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047eb93-5f95-472b-a977-ad328febc6c2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:22.338192 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047eb93-5f95-472b-a977-ad328febc6c2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:22.345220 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09ecfd-b3ed-4bba-9c68-6b689c3652bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:22.345819 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09ecfd-b3ed-4bba-9c68-6b689c3652bb admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:22.346098 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09ecfd-b3ed-4bba-9c68-6b689c3652bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:22.346368 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09ecfd-b3ed-4bba-9c68-6b689c3652bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.347009 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09ecfd-b3ed-4bba-9c68-6b689c3652bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:22.348589 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09ecfd-b3ed-4bba-9c68-6b689c3652bb admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:22.360011 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1447/2910] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:15:22.364278 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c97e43-11b2-4fd7-9daa-164f421686c9 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:22.374864 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c97e43-11b2-4fd7-9daa-164f421686c9 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:22.379432 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2097d-2e1b-4a44-b029-a9ea33539960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:22.380062 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2097d-2e1b-4a44-b029-a9ea33539960 admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:22.380336 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2097d-2e1b-4a44-b029-a9ea33539960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:22.380602 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2097d-2e1b-4a44-b029-a9ea33539960 admin admin] PATH_INFO: `/v3/projects/7e3be56263b94fc38baf9f3f209a6172/users/70ee588e1c6f43fcbef47475e88859f9/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.396339 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2097d-2e1b-4a44-b029-a9ea33539960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3be56263b94fc38baf9f3f209a6172, user_id=70ee588e1c6f43fcbef47475e88859f9, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:22.399023 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2097d-2e1b-4a44-b029-a9ea33539960 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:22.412863 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1464/2911] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:15:22 2022] PUT /identity/v3/projects/7e3be56263b94fc38baf9f3f209a6172/users/70ee588e1c6f43fcbef47475e88859f9/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:22.424208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86894859-a042-445d-9e16-a596d2cc5a09 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:22.424499 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86894859-a042-445d-9e16-a596d2cc5a09 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:22.424775 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86894859-a042-445d-9e16-a596d2cc5a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.453149 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-86894859-a042-445d-9e16-a596d2cc5a09 None None] MFA Rules not processed for user `70ee588e1c6f43fcbef47475e88859f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:22.454369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-86894859-a042-445d-9e16-a596d2cc5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:22.519765 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1448/2912] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:15:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:22.531143 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2097d-2e1b-4a44-b029-a9ea33539960 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:22.532814 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5aa2097d-2e1b-4a44-b029-a9ea33539960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:22.564019 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:22.592350 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2097d-2e1b-4a44-b029-a9ea33539960 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:22.595899 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41230a-9299-4151-870c-2a7be317e355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:22.596493 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41230a-9299-4151-870c-2a7be317e355 service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:22.596766 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41230a-9299-4151-870c-2a7be317e355 service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:22.597031 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41230a-9299-4151-870c-2a7be317e355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:22.605178 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41230a-9299-4151-870c-2a7be317e355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:22.606681 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41230a-9299-4151-870c-2a7be317e355 service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:22.615520 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1465/2913] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:32.516827 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:32.611526 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=14 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:38.818833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86894859-a042-445d-9e16-a596d2cc5a09 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:38.880892 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86894859-a042-445d-9e16-a596d2cc5a09 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:38.885560 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:38.886178 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:38.886472 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:38.886735 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] PATH_INFO: `/v3/users/70ee588e1c6f43fcbef47475e88859f9` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:38.887854 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70ee588e1c6f43fcbef47475e88859f9)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:38.889952 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:38.935905 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70ee588e1c6f43fcbef47475e88859f9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:38.942461 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70ee588e1c6f43fcbef47475e88859f9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:38.943131 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70ee588e1c6f43fcbef47475e88859f9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:38.943915 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70ee588e1c6f43fcbef47475e88859f9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:38.956041 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70ee588e1c6f43fcbef47475e88859f9'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:38.969699 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1449/2914] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:15:38 2022] DELETE /identity/v3/users/70ee588e1c6f43fcbef47475e88859f9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:38.974532 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8481b-3aff-443b-ac8f-aa2f574ccb99 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:38.974988 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8481b-3aff-443b-ac8f-aa2f574ccb99 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:38.975315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8481b-3aff-443b-ac8f-aa2f574ccb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:38.995098 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-53c8481b-3aff-443b-ac8f-aa2f574ccb99 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:38.996397 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-53c8481b-3aff-443b-ac8f-aa2f574ccb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:39.056771 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1466/2915] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:39.067066 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:39.068539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:39.126313 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399715e-cc8b-463c-9445-9a9e6b254c37 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:39.129961 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d6aad-7498-4a5f-a1dd-71a94a856d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:39.130551 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6aad-7498-4a5f-a1dd-71a94a856d45 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.130826 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6aad-7498-4a5f-a1dd-71a94a856d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.131116 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6aad-7498-4a5f-a1dd-71a94a856d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.140369 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d6aad-7498-4a5f-a1dd-71a94a856d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:39.141808 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d6aad-7498-4a5f-a1dd-71a94a856d45 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:39.151925 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1450/2916] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:39.421591 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8481b-3aff-443b-ac8f-aa2f574ccb99 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:39.423887 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-53c8481b-3aff-443b-ac8f-aa2f574ccb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:39.446241 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8481b-3aff-443b-ac8f-aa2f574ccb99 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:39.452637 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:39.453203 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.453509 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.453801 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] PATH_INFO: `/v3/projects/7e3be56263b94fc38baf9f3f209a6172` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.454885 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3be56263b94fc38baf9f3f209a6172)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:39.456810 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:39.504392 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3be56263b94fc38baf9f3f209a6172', 'request_id': 'req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:39.510650 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3be56263b94fc38baf9f3f209a6172', 'request_id': 'req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:39.518076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3be56263b94fc38baf9f3f209a6172', 'request_id': 'req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:39.528629 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] The token cache is being invalidate because project 7e3be56263b94fc38baf9f3f209a6172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:15:39.528938 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:39.529670 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-bd853e42-ac53-48ec-93bb-fe9b8ba0706f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:39.531030 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1467/2917] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:15:39 2022] DELETE /identity/v3/projects/7e3be56263b94fc38baf9f3f209a6172 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:39.535948 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b424c3-f791-4018-9b7b-edbaa2645073 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.536231 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b424c3-f791-4018-9b7b-edbaa2645073 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.536501 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b424c3-f791-4018-9b7b-edbaa2645073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.556058 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-53b424c3-f791-4018-9b7b-edbaa2645073 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:39.557254 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-53b424c3-f791-4018-9b7b-edbaa2645073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:39.575075 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:39.618335 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1451/2918] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:39.626946 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaacfb3-d027-48ce-b28a-17adbad4a908 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.627229 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaacfb3-d027-48ce-b28a-17adbad4a908 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.627497 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaacfb3-d027-48ce-b28a-17adbad4a908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.646809 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-5aaacfb3-d027-48ce-b28a-17adbad4a908 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:39.647991 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-5aaacfb3-d027-48ce-b28a-17adbad4a908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:39.656954 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1468/2919] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:39.661175 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b424c3-f791-4018-9b7b-edbaa2645073 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:39.670994 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b424c3-f791-4018-9b7b-edbaa2645073 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:39.675483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606f026-4854-4594-b47e-6d760b50827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:39.676069 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f026-4854-4594-b47e-6d760b50827d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.676346 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f026-4854-4594-b47e-6d760b50827d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.676612 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f026-4854-4594-b47e-6d760b50827d admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.677281 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606f026-4854-4594-b47e-6d760b50827d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:15:39.677583 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606f026-4854-4594-b47e-6d760b50827d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:39.679143 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606f026-4854-4594-b47e-6d760b50827d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:39.691462 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1452/2920] 10.222.0.78 () {58 vars in 1153 bytes} [Tue Aug 30 01:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:15:39.695654 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaacfb3-d027-48ce-b28a-17adbad4a908 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:39.705607 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaacfb3-d027-48ce-b28a-17adbad4a908 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:39.710233 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6141d35-572b-4099-834e-0a82a8d0e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:39.710821 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6141d35-572b-4099-834e-0a82a8d0e261 admin admin] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.711096 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6141d35-572b-4099-834e-0a82a8d0e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.711365 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6141d35-572b-4099-834e-0a82a8d0e261 admin admin] PATH_INFO: `/v3/projects` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.711987 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6141d35-572b-4099-834e-0a82a8d0e261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:39.713923 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6141d35-572b-4099-834e-0a82a8d0e261 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:39.740360 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1469/2921] 10.222.0.78 () {60 vars in 1153 bytes} [Tue Aug 30 01:15:39 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:15:39.744657 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606f026-4854-4594-b47e-6d760b50827d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:39.756557 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606f026-4854-4594-b47e-6d760b50827d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:39.813381 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a51402-a368-4e1d-a877-18de4e4b39c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:39.813973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a51402-a368-4e1d-a877-18de4e4b39c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.814251 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a51402-a368-4e1d-a877-18de4e4b39c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.814521 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a51402-a368-4e1d-a877-18de4e4b39c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.815135 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a51402-a368-4e1d-a877-18de4e4b39c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:39.817036 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a51402-a368-4e1d-a877-18de4e4b39c8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:39.840349 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1453/2922] 10.222.0.78 () {60 vars in 1144 bytes} [Tue Aug 30 01:15:39 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:15:39.845218 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6141d35-572b-4099-834e-0a82a8d0e261 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:39.855470 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6141d35-572b-4099-834e-0a82a8d0e261 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:39.859967 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7ad42-9e9b-4992-99e1-39f4ec70e4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:39.860558 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7ad42-9e9b-4992-99e1-39f4ec70e4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.860836 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7ad42-9e9b-4992-99e1-39f4ec70e4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.861130 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7ad42-9e9b-4992-99e1-39f4ec70e4b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.861769 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7ad42-9e9b-4992-99e1-39f4ec70e4b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:39.863353 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7ad42-9e9b-4992-99e1-39f4ec70e4b2 admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:39.874675 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1470/2923] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:15:39.878592 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a51402-a368-4e1d-a877-18de4e4b39c8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:39.889183 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a51402-a368-4e1d-a877-18de4e4b39c8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:39.893833 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d11f00-b834-4fed-9062-d71a0cf51427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:39.894420 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d11f00-b834-4fed-9062-d71a0cf51427 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.894702 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d11f00-b834-4fed-9062-d71a0cf51427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.894973 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d11f00-b834-4fed-9062-d71a0cf51427 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.895611 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d11f00-b834-4fed-9062-d71a0cf51427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:39.897228 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d11f00-b834-4fed-9062-d71a0cf51427 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:39.908252 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1454/2924] 10.222.0.78 () {58 vars in 1122 bytes} [Tue Aug 30 01:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:15:39.912190 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7ad42-9e9b-4992-99e1-39f4ec70e4b2 admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:39.922435 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7ad42-9e9b-4992-99e1-39f4ec70e4b2 admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:39.927475 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec9f3a-a4dc-4138-880a-eff7ae1dc39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:39.928302 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec9f3a-a4dc-4138-880a-eff7ae1dc39b admin admin] REQUEST_METHOD: `PUT` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.928592 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec9f3a-a4dc-4138-880a-eff7ae1dc39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.928898 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec9f3a-a4dc-4138-880a-eff7ae1dc39b admin admin] PATH_INFO: `/v3/projects/fc1efce28a7b473cb8174749c460884c/users/b719fb17cc264d91911f1d9393e97a06/roles/e407aec03c22425591fa0e52a40edbc1` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.944066 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec9f3a-a4dc-4138-880a-eff7ae1dc39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1efce28a7b473cb8174749c460884c, user_id=b719fb17cc264d91911f1d9393e97a06, role_id=e407aec03c22425591fa0e52a40edbc1)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:39.946736 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec9f3a-a4dc-4138-880a-eff7ae1dc39b admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:39.960074 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1471/2925] 10.222.0.78 () {60 vars in 1483 bytes} [Tue Aug 30 01:15:39 2022] PUT /identity/v3/projects/fc1efce28a7b473cb8174749c460884c/users/b719fb17cc264d91911f1d9393e97a06/roles/e407aec03c22425591fa0e52a40edbc1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:39.971678 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c0afc-c7c4-46a3-86b6-ec9ad9e8e400 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:39.971966 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c0afc-c7c4-46a3-86b6-ec9ad9e8e400 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:39.972243 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c0afc-c7c4-46a3-86b6-ec9ad9e8e400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:39.999910 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-089c0afc-c7c4-46a3-86b6-ec9ad9e8e400 None None] MFA Rules not processed for user `b719fb17cc264d91911f1d9393e97a06`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:40.001160 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-089c0afc-c7c4-46a3-86b6-ec9ad9e8e400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:40.064481 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1455/2926] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:15:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:40.075657 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec9f3a-a4dc-4138-880a-eff7ae1dc39b admin admin] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:40.078167 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-77ec9f3a-a4dc-4138-880a-eff7ae1dc39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:40.100044 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:40.136939 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec9f3a-a4dc-4138-880a-eff7ae1dc39b admin admin] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:40.141193 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985d655-586d-4907-bf59-7f3d1b0695df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e64ee3c8ee48c2ad8f351fb2a78bc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:40.141796 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985d655-586d-4907-bf59-7f3d1b0695df service cinder] REQUEST_METHOD: `GET` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:40.142076 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985d655-586d-4907-bf59-7f3d1b0695df service cinder] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:40.142347 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985d655-586d-4907-bf59-7f3d1b0695df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:40.150755 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985d655-586d-4907-bf59-7f3d1b0695df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:40.152179 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985d655-586d-4907-bf59-7f3d1b0695df service cinder] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:40.161614 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1472/2927] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:40 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:47.389293 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c0afc-c7c4-46a3-86b6-ec9ad9e8e400 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:47.408539 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:47.453931 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c0afc-c7c4-46a3-86b6-ec9ad9e8e400 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:47.458388 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:47.459006 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:47.459279 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:47.459548 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] PATH_INFO: `/v3/users/b719fb17cc264d91911f1d9393e97a06` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:47.460655 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b719fb17cc264d91911f1d9393e97a06)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:47.462568 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:47.508879 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b719fb17cc264d91911f1d9393e97a06'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:47.515858 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b719fb17cc264d91911f1d9393e97a06'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:47.516483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b719fb17cc264d91911f1d9393e97a06'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:47.517114 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b719fb17cc264d91911f1d9393e97a06'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:47.528970 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b719fb17cc264d91911f1d9393e97a06'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:47.540941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1456/2928] 10.222.0.78 () {58 vars in 1224 bytes} [Tue Aug 30 01:15:47 2022] DELETE /identity/v3/users/b719fb17cc264d91911f1d9393e97a06 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:47.546007 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6deca-4535-44a3-99c8-78694792fec7 None None] REQUEST_METHOD: `POST` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:47.546521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6deca-4535-44a3-99c8-78694792fec7 None None] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:47.546521 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6deca-4535-44a3-99c8-78694792fec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:47.565315 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.auth.core [None req-a7a6deca-4535-44a3-99c8-78694792fec7 None None] MFA Rules not processed for user `d9331e55306b44e2a9e5ecea982fbc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81443) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:47.566517 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a7a6deca-4535-44a3-99c8-78694792fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:47.624791 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1473/2929] 10.222.0.78 () {58 vars in 958 bytes} [Tue Aug 30 01:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:47.634627 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:47.636046 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:47.693096 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54e4c2-d692-43e5-8065-aa2382189114 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:47.696781 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e089dd-83c1-4a34-a3d0-f07801e92fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a5d841380c243209f55314b545ff56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:47.697402 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e089dd-83c1-4a34-a3d0-f07801e92fed service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:47.697680 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e089dd-83c1-4a34-a3d0-f07801e92fed service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:47.697943 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e089dd-83c1-4a34-a3d0-f07801e92fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:47.707585 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e089dd-83c1-4a34-a3d0-f07801e92fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:47.709021 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e089dd-83c1-4a34-a3d0-f07801e92fed service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:47.719569 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1457/2930] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:15:47.985537 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6deca-4535-44a3-99c8-78694792fec7 None None] Authenticating user token {{(pid=81443) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:47.987829 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.fernet_utils [None req-a7a6deca-4535-44a3-99c8-78694792fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81443) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:48.001474 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6deca-4535-44a3-99c8-78694792fec7 None None] Validating token access rules against request {{(pid=81443) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:48.006108 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9331e55306b44e2a9e5ecea982fbc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '761e332e13aa43bb851c08b247de0d02', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81443) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:48.006737 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:48.007151 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:48.007478 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] PATH_INFO: `/v3/projects/fc1efce28a7b473cb8174749c460884c` {{(pid=81443) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:48.008604 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1efce28a7b473cb8174749c460884c)` {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:48.010548 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] RBAC: Authorization granted {{(pid=81443) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:48.059501 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1efce28a7b473cb8174749c460884c', 'request_id': 'req-8d9ebce1-468c-4213-a678-1b48cda62f0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:48.066220 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1efce28a7b473cb8174749c460884c', 'request_id': 'req-8d9ebce1-468c-4213-a678-1b48cda62f0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:48.073700 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1efce28a7b473cb8174749c460884c', 'request_id': 'req-8d9ebce1-468c-4213-a678-1b48cda62f0f'} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:48.084720 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] The token cache is being invalidate because project fc1efce28a7b473cb8174749c460884c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81443) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:15:48.085011 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:48.085715 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG keystone.notifications [None req-8d9ebce1-468c-4213-a678-1b48cda62f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81443) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:48.087264 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: [pid: 81443|app: 0|req: 1474/2931] 10.222.0.78 () {58 vars in 1233 bytes} [Tue Aug 30 01:15:47 2022] DELETE /identity/v3/projects/fc1efce28a7b473cb8174749c460884c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:57.714639 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:58.081524 dv-cinder-815146-13-iscsi devstack@keystone.service[81443]: DEBUG dbcounter [-] [81443] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81443) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:31.425351 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e089dd-83c1-4a34-a3d0-f07801e92fed service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:31.426768 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-50e089dd-83c1-4a34-a3d0-f07801e92fed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:31.573373 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e089dd-83c1-4a34-a3d0-f07801e92fed service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:31.631334 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ad834-2ab4-44dd-81c7-ff4d0f26602f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f860912e1f0406cadd70da440b2ede2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e40bb661d1a4f5cadf826e3a93af104', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:31.631930 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ad834-2ab4-44dd-81c7-ff4d0f26602f service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:31.632208 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ad834-2ab4-44dd-81c7-ff4d0f26602f service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:31.632483 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ad834-2ab4-44dd-81c7-ff4d0f26602f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:31.634251 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c92ad834-2ab4-44dd-81c7-ff4d0f26602f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:31.645397 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ad834-2ab4-44dd-81c7-ff4d0f26602f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:31.646941 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ad834-2ab4-44dd-81c7-ff4d0f26602f service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:31.657454 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1458/2932] 10.222.0.78 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:41.653017 dv-cinder-815146-13-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}